-- Logs begin at Mon 2022-08-29 22:56:15 UTC, end at Tue 2022-08-30 00:13:00 UTC. -- Aug 29 23:28:49.550972 dv-cinder-854272-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 29 23:28:49.560068 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 29 23:28:49 2022] *** Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: nodename: dv-cinder-854272-4-iscsi Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: machine: x86_64 Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: clock source: unix Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: pcre jit disabled Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: detected number of CPU cores: 4 Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: current working directory: / Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: detected binary path: /usr/bin/uwsgi-core Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: your processes number limit is 47680 Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: your memory page size is 4096 bytes Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: detected max file descriptor number: 2048 Aug 29 23:28:49.562207 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: lock engine: pthread robust mutexes Aug 29 23:28:49.563473 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: thunder lock: enabled Aug 29 23:28:49.563473 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 29 23:28:49.563473 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 29 23:28:49.604067 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: Python main interpreter initialized at 0x563bddf2e9c0 Aug 29 23:28:49.604067 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: python threads support enabled Aug 29 23:28:49.604067 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: your server socket listen backlog is limited to 100 connections Aug 29 23:28:49.604067 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: your mercy for graceful operations on workers is 90 seconds Aug 29 23:28:49.604457 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: mapped 403077 bytes (393 KB) for 2 cores Aug 29 23:28:49.604457 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: *** Operational MODE: preforking *** Aug 29 23:28:49.604994 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: *** uWSGI is running in multiple interpreter mode *** Aug 29 23:28:49.604994 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: spawned uWSGI master process (pid: 81839) Aug 29 23:28:49.605248 dv-cinder-854272-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 29 23:28:49.605836 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: spawned uWSGI worker 1 (pid: 81852, cores: 1) Aug 29 23:28:49.606118 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: spawned uWSGI worker 2 (pid: 81853, cores: 1) Aug 29 23:28:49.608739 dv-cinder-854272-4-iscsi devstack@keystone.service[81839]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 29 23:28:54.159785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 23:28:54.159785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: from cryptography.utils import int_from_bytes Aug 29 23:28:54.160663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 23:28:54.160663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: from cryptography.utils import int_from_bytes Aug 29 23:28:54.160799 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 23:28:54.160799 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: from cryptography.utils import int_from_bytes Aug 29 23:28:54.160799 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 23:28:54.160799 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: from cryptography.utils import int_from_bytes Aug 29 23:28:54.217049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 23:28:54.217280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 23:28:54.217400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] command line args: [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 23:28:54.217673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 23:28:54.218409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ================================================================================ {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 23:28:54.218702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] admin_token = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.218951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] config_dir = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.219141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.219332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] config_source = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.219515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] debug = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.219709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.220141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.220272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 23:28:54.220392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.220494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 23:28:54.220494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] command line args: [] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 23:28:54.220686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.220790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.220891 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 23:28:54.220891 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ================================================================================ {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 23:28:54.221095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.221095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_config_append = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.221280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] admin_token = **** {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.221383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.221494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] config_dir = [] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.221610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_dir = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.221710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.221808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_file = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.221914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] config_source = [] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.222026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.222131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] debug = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.222241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.222338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.222509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.222625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.222625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.222822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.222822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223039 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223132 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223223 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] list_limit = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] log_config_append = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223565 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223661 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] log_dir = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223910 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.223981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] log_file = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.224066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.224153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.224296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.224453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.224567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.224674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.224782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.224890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.225009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.225126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.225239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.225362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.225481 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.225618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.225736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] publish_errors = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.225859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.225974 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.226089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.226326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.226478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.226604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.226719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.226831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.226957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.227085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.227201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.227328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] standard_threads = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.227328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.227600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.227706 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.227706 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.228019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.228134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] use_journal = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.228270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] publish_errors = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.228384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] use_json = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.228384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] use_stderr = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.228608 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.228736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] use_syslog = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.228859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.228995 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.229110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.229283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.229394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.229522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.229635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.229635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] standard_threads = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.229858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.229978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.230102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.230227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.230348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.230506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.230613 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.230732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] use_journal = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.230732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] use_json = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.231050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.231202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] use_stderr = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.231397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.231506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] use_syslog = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.231631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.231736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 23:28:54.231862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.231983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.232097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.232210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.232328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.232434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.232544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.232663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.232778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.232884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.233004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.233122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.233238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.233360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.233466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.233575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.233689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.233808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.233914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.234043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.234148 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.234258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.234375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.234501 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.234632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.234752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.234859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.234968 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.235076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.235184 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.235315 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.235439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.235543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.235658 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.235658 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.235878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.235987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.236113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.236227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.236333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.236443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.236550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.236660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.236970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.237078 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.237190 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.237479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.237589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.237699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.237822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.237945 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.238055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.238215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.238321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.238506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.238631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.238742 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.238862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.238968 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.239077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.239234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.239354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.239468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.239576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.239687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.239804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.239924 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.240039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.240155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.240261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.240407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.240550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.240714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.240866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.241017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.241165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.241310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.241507 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.241668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.241832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.241988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.242154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.242281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.242473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.242612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.242787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.242953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.243065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.243252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.243462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.243609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.243725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.243841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.243963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.244068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.244177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.244293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.244440 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.244547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.244656 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.244762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.244879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.245031 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.245154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.245265 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.245371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.245478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.245585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.245707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.245849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.245969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.246077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.246191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.246319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.246494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.246621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.248132 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.248295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.248411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.248525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.248686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.248811 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.248918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.249028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.249144 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.249309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.249415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.249527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.249650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.249758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.249878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.250008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.250134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.250242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.250365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.250496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.250614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.250991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.251101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.251212 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.251405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.251513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.251625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.251735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.251845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.251955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.252077 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.252184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.252295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.252406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.252527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.252647 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.252790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.252923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.253032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.253142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.253253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.253364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.253474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.253474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.253701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.253808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.253808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.254022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.254137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.254264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.254371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.254518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.254634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.254913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.254913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.255139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.255257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.255364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.255474 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.255586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.255700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.255827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.255945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.256060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.256174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.256289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.256403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.256531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.256642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.256765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.256877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.256997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.257109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.257235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.257355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.257467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.257585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.257700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.257814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.257939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.257939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.258146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.258259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.258386 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.external = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.258586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.258786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.258897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.259109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.259288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.259430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.259643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.259764 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.password = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.259933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.260065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.token = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.260178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.260304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.260416 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] auth.external = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.260534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.260651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.260780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.260780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.260989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.261112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.261236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.261236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] auth.password = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.261474 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.261662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] auth.token = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.261782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.261902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.262073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.262197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.262391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.262573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.262794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.262906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.263159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.263282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.263525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.263747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.263866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.264111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.264222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.264475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] credential.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.264589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.265423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.265533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.265732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.266177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.266332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.266462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.266592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.266706 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.266843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.266953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.267087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.267197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.267314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.267431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.267556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.267680 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.267873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.267983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.268099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.268215 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.268336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.268451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.268579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.268692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.268821 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.268932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.269058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.269178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.269312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.269470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.269579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] federation.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.269696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.269836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.269967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.270091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.270201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.270317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.270509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.270619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.270737 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.270870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.270979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.271097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.271210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.271328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.271443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.271571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.271687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.271687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.271901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.272014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.272129 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.272129 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.272354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.272470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.272585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.272699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.272814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.273035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.273147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.273270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.273385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.273501 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.273615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.273615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.273840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.273840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.274052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.274165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.274292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.274292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.274645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.274757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.274878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.275000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.275130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.275256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.275371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.275490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.275609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.275726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.275857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.275971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.276089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.276205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.276325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.276444 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.276579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.276952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.277071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.277190 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.277308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.277426 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.277554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.277751 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.277876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.277993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.278112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.278231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.278360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.278518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.278636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.278751 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.278869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.278984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.279116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.279234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.279348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.279462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.279572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.279689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.279817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.279929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.280046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.280167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.280307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.280952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.281083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.281195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.281312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.281428 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.281544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.281658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.281785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.281931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.282199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.282310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.282445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.282558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.282690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.282806 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.282925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.283043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.283161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.283305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.283430 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.283546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.283664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.283878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.283995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.284113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.284245 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.284396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.284506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.284627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.284745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.284863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.284996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.285111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.285230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.285349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.285467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.285584 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.285713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.285829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.285954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.286071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.286189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.286306 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.286454 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.286573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.286691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.286807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.286923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.287057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.287168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.287295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.287407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.287524 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.287644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.287771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.287882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.287985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.288094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.288094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.288293 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.288430 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.288545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.288661 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.288789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.288907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.289020 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.289150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.289308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.289425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.289540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.289650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.289771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.289901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.290018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.290137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.290258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.290376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.290547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.290671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.290787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.290911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.291028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.291147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.291264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.291392 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.291509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.291636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.291753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.291871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.291990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.292202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.292313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.292434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.292552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.292675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.292793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.292922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.293038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.293156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.293274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.293393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.293510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.293640 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.293756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.293877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.293877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.294089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.294206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.294343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.294477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.294597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.294711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.294826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.294940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] resource.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.295073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.295230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.295406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.295554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.295655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.295752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.295859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.296020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.296119 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] role.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.296216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.296313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] role.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.296467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.296674 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.296774 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.296873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.296969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.297076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.297174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.297283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.297381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] role.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.297477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.297574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] role.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.297670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.297768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.297872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.297970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.298067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.298163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.298259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.298356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.302371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.302688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.302801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.302923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.303046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.303176 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.303292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.303412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.303530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.303651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.303651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.303883 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.303996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.304114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.304232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.304352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.304470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.304599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.304719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.304838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.304838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.305059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.305177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.305306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.305420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.305538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.305656 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.305775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.305931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.306071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.306182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.306301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.306435 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.306557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.306676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.306815 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.306924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.307042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.307158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.307275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.307391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.307522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.307634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.307751 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.307871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.307986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.308102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.308230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.308343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.308460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.308577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.308694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.308809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.308938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.309050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.309167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.309282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.309399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.309533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.309645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] token.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.309804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.309913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.310033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.310150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.310281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.310397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.310650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.310780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.310906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.311023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.311162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.311281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.311400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.311571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.311685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.311805 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.312077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.312195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.312326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.312454 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.312582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.312711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.312830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.312954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.313077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.connection = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.313222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.313342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.313462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.313581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.313699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.313838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.313971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.314085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.314205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.314324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.314462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.connection = **** {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.314587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.314722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.314833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.314957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.315073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.315190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.315307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.315508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.315619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.315738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.315853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.315970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.316088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.316219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.316332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.316449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.316564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.316686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.316801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.316931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.317042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.317161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.317262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.317373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.317595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.317724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.317949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 23:28:54.318114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.318114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.318114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.318114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.318114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 23:28:54.318114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81853) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 23:28:54.338985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.339255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.339434 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.339608 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.339748 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.339910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.349156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.349280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.378712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.378953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.379133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.379254 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.436186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.436624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.498367 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.498625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.499139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.499401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.662370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.662642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.662821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.663051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.663274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.663486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.663635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.663760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.663930 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.664366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.664586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.688217 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.688425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.688564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.688785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.688927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.689068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.689194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.689317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.689433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.689562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.689678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.689802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.689923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.690044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.701032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.701207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.714060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.714234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.714305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.714368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.714519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.714634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.714744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.714864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.714949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.715085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.715171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.715319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.715494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81853) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.715638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 23:28:54.733042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.733256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.733377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 23:28:54.734073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 23:28:54.734192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.734377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.734560 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 23:28:54.734680 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 23:28:54.734807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.734994 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.735111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.735255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.735526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.735652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.735774 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.735892 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.745130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.745409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.745526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.745680 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.745797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.745968 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.746100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.746266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.752953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.753516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.755853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.756135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.756449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.756568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.756836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.756968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.757086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.757181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.757249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.757310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.757476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.757616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.757762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.757865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.758063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.758157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.758341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.758523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.758643 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.758774 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.758920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.759199 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.783364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.783603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.784864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.784989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.785151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.785269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.789738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.789875 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.790036 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.790156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.790323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.790478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.797508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.797802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.797887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.798163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.801776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.802075 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.802240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.802368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.802554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.802683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.802840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.802965 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.803123 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.803431 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.803724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.803995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.804264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.805121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.805420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.805714 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.821323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.821616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.823416 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.823759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.825493 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.829159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.829366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.829450 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.832838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.833135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.833762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.836890 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.837338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.839980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.840289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.840576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.840668 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.840862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.841146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.841540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.844486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.844788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.845104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.845439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.845737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.846018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.858663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.858964 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.859433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.859705 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.862148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.862461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.862746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.863010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.866505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.868180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.869731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.869871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.869999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.871407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.872663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.872935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.873071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.873222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.873341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.873523 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.873816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.875958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.876252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.876649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.876935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.877223 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.885317 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.885609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.885898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.886185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.886499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.886787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.887086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.888378 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.888665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.889049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.889329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.890171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.890519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.890821 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.903419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.907146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.907307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.910201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.911289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.912766 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.913282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.914266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.914589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.914813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.914936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.916434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.916768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.917016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.921651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.923182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.924038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.925519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.926070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.926375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.926695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.926983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.928635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.928932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.929213 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.929486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.935009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.935327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.935599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.938189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.938612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.938791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.943996 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.944505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.944991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.945467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.945839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81853) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 23:28:54.945964 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.946139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.946388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.946577 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.946718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.946994 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.947109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.947270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.947546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.947664 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.947827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.948079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.948198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.948383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.948573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.948692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.948905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.949049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.949181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.949434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81853) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.949547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.949955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.950472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 23:28:54.992396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563bddf2e9c0 pid: 81852 (default app) Aug 29 23:28:54.996320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc1e4d-24de-4f59-913c-206b74b271a9 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:28:54.996612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc1e4d-24de-4f59-913c-206b74b271a9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:28:54.996891 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc1e4d-24de-4f59-913c-206b74b271a9 None None] PATH_INFO: `/v3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:28:54.997096 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563bddf2e9c0 pid: 81853 (default app) Aug 29 23:28:55.001496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Mon Aug 29 23:28:54 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:02.654912 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e9d79-b169-4c4e-8a5b-21a5935dac24 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:02.655571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e9d79-b169-4c4e-8a5b-21a5935dac24 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:02.655571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e9d79-b169-4c4e-8a5b-21a5935dac24 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:02.659704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1/2] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:29:02.665522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127ad36-4e31-49c1-91a6-7cb02829adc0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:02.677071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127ad36-4e31-49c1-91a6-7cb02829adc0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:02.677071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127ad36-4e31-49c1-91a6-7cb02829adc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:02.737589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: INFO dbcounter [None req-d127ad36-4e31-49c1-91a6-7cb02829adc0 None None] Registered counter for database keystone Aug 29 23:29:02.756935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG oslo_db.sqlalchemy.engines [None req-d127ad36-4e31-49c1-91a6-7cb02829adc0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81852) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 23:29:02.763327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writer thread running {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 23:29:02.949928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG passlib.handlers.bcrypt [None req-d127ad36-4e31-49c1-91a6-7cb02829adc0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81852) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 23:29:02.950244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG passlib.handlers.bcrypt [None req-d127ad36-4e31-49c1-91a6-7cb02829adc0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81852) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 23:29:03.009624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d127ad36-4e31-49c1-91a6-7cb02829adc0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:03.011270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d127ad36-4e31-49c1-91a6-7cb02829adc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:03.144296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 2/3] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 23:29:03.605717 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e9d79-b169-4c4e-8a5b-21a5935dac24 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:03.677091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: INFO dbcounter [None req-351e9d79-b169-4c4e-8a5b-21a5935dac24 None None] Registered counter for database keystone Aug 29 23:29:03.698920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG oslo_db.sqlalchemy.engines [None req-351e9d79-b169-4c4e-8a5b-21a5935dac24 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81853) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 23:29:03.705587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writer thread running {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 23:29:03.828600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e9d79-b169-4c4e-8a5b-21a5935dac24 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:03.832918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee4f0d-8469-4303-af04-6ea39e21ce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:03.833552 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee4f0d-8469-4303-af04-6ea39e21ce77 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:03.833831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee4f0d-8469-4303-af04-6ea39e21ce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:03.834103 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee4f0d-8469-4303-af04-6ea39e21ce77 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:03.844191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee4f0d-8469-4303-af04-6ea39e21ce77 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:03.896951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: WARNING keystone.common.rbac_enforcer.enforcer [None req-66ee4f0d-8469-4303-af04-6ea39e21ce77 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 23:29:03.897209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee4f0d-8469-4303-af04-6ea39e21ce77 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:03.908541 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 2/4] 10.222.0.58 () {58 vars in 1324 bytes} [Mon Aug 29 23:29:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 23:29:03.914866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127ad36-4e31-49c1-91a6-7cb02829adc0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:03.930839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127ad36-4e31-49c1-91a6-7cb02829adc0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:03.935513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053bc21-1827-493e-9bbe-bdd71c73bd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:03.936683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053bc21-1827-493e-9bbe-bdd71c73bd1a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:03.936966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053bc21-1827-493e-9bbe-bdd71c73bd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:03.937237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053bc21-1827-493e-9bbe-bdd71c73bd1a admin admin] PATH_INFO: `/v3/projects/291cfd89a4d34a9da9d99e5ca0c9d4dd` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:03.938908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053bc21-1827-493e-9bbe-bdd71c73bd1a admin admin] RBAC: Authorizing `identity:get_project(project_id=291cfd89a4d34a9da9d99e5ca0c9d4dd)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:03.994039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6053bc21-1827-493e-9bbe-bdd71c73bd1a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 23:29:03.994381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053bc21-1827-493e-9bbe-bdd71c73bd1a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:03.999568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 3/5] 10.222.0.58 () {56 vars in 1207 bytes} [Mon Aug 29 23:29:03 2022] GET /identity/v3/projects/291cfd89a4d34a9da9d99e5ca0c9d4dd => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:05.954640 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e1255-bfa4-49d8-9367-afc02ecf12dc None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:05.955299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e1255-bfa4-49d8-9367-afc02ecf12dc None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:05.955299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e1255-bfa4-49d8-9367-afc02ecf12dc None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:05.959394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 3/6] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:05.965352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb9c44-019e-4cd0-97f4-dab4e9314a9e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:05.965899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb9c44-019e-4cd0-97f4-dab4e9314a9e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:05.965899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb9c44-019e-4cd0-97f4-dab4e9314a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:05.987653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-eceb9c44-019e-4cd0-97f4-dab4e9314a9e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:05.989217 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-eceb9c44-019e-4cd0-97f4-dab4e9314a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:05.997015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 4/7] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 23:29:06.269726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e1255-bfa4-49d8-9367-afc02ecf12dc None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:06.284447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e1255-bfa4-49d8-9367-afc02ecf12dc None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:06.289084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e14f4d-8765-4c0f-971d-1e072dcd6815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:06.289787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e14f4d-8765-4c0f-971d-1e072dcd6815 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:06.290140 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e14f4d-8765-4c0f-971d-1e072dcd6815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:06.290546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e14f4d-8765-4c0f-971d-1e072dcd6815 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:06.300033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e14f4d-8765-4c0f-971d-1e072dcd6815 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:06.301915 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e14f4d-8765-4c0f-971d-1e072dcd6815 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:06.310975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 4/8] 10.222.0.58 () {58 vars in 1324 bytes} [Mon Aug 29 23:29:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:29:06.316811 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb9c44-019e-4cd0-97f4-dab4e9314a9e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:06.327800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb9c44-019e-4cd0-97f4-dab4e9314a9e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:06.332325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57febf04-d85d-4112-a5e6-d103a5c609c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:06.332973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57febf04-d85d-4112-a5e6-d103a5c609c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:06.333255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57febf04-d85d-4112-a5e6-d103a5c609c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:06.333532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57febf04-d85d-4112-a5e6-d103a5c609c6 admin admin] PATH_INFO: `/v3/users/557208629c7f4826b594ef28cabef8ea` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:06.334896 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57febf04-d85d-4112-a5e6-d103a5c609c6 admin admin] RBAC: Authorizing `identity:get_user(user_id=557208629c7f4826b594ef28cabef8ea)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:06.336767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57febf04-d85d-4112-a5e6-d103a5c609c6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:06.338532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 5/9] 10.222.0.58 () {56 vars in 1198 bytes} [Mon Aug 29 23:29:06 2022] GET /identity/v3/users/557208629c7f4826b594ef28cabef8ea => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:08.373122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441c8ca-2e5c-4ee4-8c87-60e165175a17 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:08.373779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441c8ca-2e5c-4ee4-8c87-60e165175a17 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:08.373779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441c8ca-2e5c-4ee4-8c87-60e165175a17 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:08.375102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 5/10] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:08.381109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae01104-9bb1-472d-bd2b-9834952a2b04 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:08.381651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae01104-9bb1-472d-bd2b-9834952a2b04 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:08.381733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae01104-9bb1-472d-bd2b-9834952a2b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:08.402056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5ae01104-9bb1-472d-bd2b-9834952a2b04 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:08.403322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5ae01104-9bb1-472d-bd2b-9834952a2b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:08.409401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba3150-44f4-4916-8cb7-361c89a04744 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:08.409813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba3150-44f4-4916-8cb7-361c89a04744 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:08.410186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba3150-44f4-4916-8cb7-361c89a04744 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:08.411681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 6/11] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:08.417640 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de33754-b741-408a-bbd3-bbbb92ea4e04 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:08.417924 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de33754-b741-408a-bbd3-bbbb92ea4e04 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:08.418261 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de33754-b741-408a-bbd3-bbbb92ea4e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:08.462370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG passlib.handlers.bcrypt [None req-1de33754-b741-408a-bbd3-bbbb92ea4e04 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81853) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 23:29:08.462827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG passlib.handlers.bcrypt [None req-1de33754-b741-408a-bbd3-bbbb92ea4e04 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81853) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 23:29:08.492271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 6/12] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:08.512495 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-1de33754-b741-408a-bbd3-bbbb92ea4e04 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:08.514282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1de33754-b741-408a-bbd3-bbbb92ea4e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:08.621527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 7/13] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:29:08.763801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae01104-9bb1-472d-bd2b-9834952a2b04 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:08.789036 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:08.875763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae01104-9bb1-472d-bd2b-9834952a2b04 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:08.894407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de33754-b741-408a-bbd3-bbbb92ea4e04 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:08.913298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c66e3-0015-4983-b88e-4e4abf58fea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:08.913954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c66e3-0015-4983-b88e-4e4abf58fea5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:08.914234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c66e3-0015-4983-b88e-4e4abf58fea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:08.914545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c66e3-0015-4983-b88e-4e4abf58fea5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:08.927651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c66e3-0015-4983-b88e-4e4abf58fea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-0f9c66e3-0015-4983-b88e-4e4abf58fea5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:08.936373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:08.937375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:08.938261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 7/14] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:29:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:08.943620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c66e3-0015-4983-b88e-4e4abf58fea5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:09.021756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de33754-b741-408a-bbd3-bbbb92ea4e04 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:09.024371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:09.045720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:09.071333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b1386-fca4-4d5c-95f5-aedfa3210266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:09.072745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b1386-fca4-4d5c-95f5-aedfa3210266 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:09.073214 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b1386-fca4-4d5c-95f5-aedfa3210266 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:09.073666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b1386-fca4-4d5c-95f5-aedfa3210266 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:09.074145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c66e3-0015-4983-b88e-4e4abf58fea5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:09.118302 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43a758-a0d9-4e73-99ee-ef5d6d1aae0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:09.119484 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43a758-a0d9-4e73-99ee-ef5d6d1aae0f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:09.119984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43a758-a0d9-4e73-99ee-ef5d6d1aae0f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:09.120490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43a758-a0d9-4e73-99ee-ef5d6d1aae0f None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:09.121757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43a758-a0d9-4e73-99ee-ef5d6d1aae0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:09.122854 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43a758-a0d9-4e73-99ee-ef5d6d1aae0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:09.125496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43a758-a0d9-4e73-99ee-ef5d6d1aae0f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:09.148748 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 8/15] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:09.155651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43a758-a0d9-4e73-99ee-ef5d6d1aae0f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:09.158477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b1386-fca4-4d5c-95f5-aedfa3210266 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:09.160034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b1386-fca4-4d5c-95f5-aedfa3210266 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:09.266498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 8/16] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 23:29:09.272112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b1386-fca4-4d5c-95f5-aedfa3210266 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:09.275030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:09.284473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43a758-a0d9-4e73-99ee-ef5d6d1aae0f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:09.319051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773badb9-ee8b-49a1-8bda-825e6edacac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:09.319691 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773badb9-ee8b-49a1-8bda-825e6edacac3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:09.319969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773badb9-ee8b-49a1-8bda-825e6edacac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:09.320239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773badb9-ee8b-49a1-8bda-825e6edacac3 None admin] PATH_INFO: `/v3/users/557208629c7f4826b594ef28cabef8ea` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:09.321429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773badb9-ee8b-49a1-8bda-825e6edacac3 None admin] RBAC: Authorizing `identity:get_user(user_id=557208629c7f4826b594ef28cabef8ea)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:09.322954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773badb9-ee8b-49a1-8bda-825e6edacac3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:09.324562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 9/17] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:09 2022] GET /identity/v3/users/557208629c7f4826b594ef28cabef8ea => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:09.330412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773badb9-ee8b-49a1-8bda-825e6edacac3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:09.353180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:09.399583 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b1386-fca4-4d5c-95f5-aedfa3210266 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:09.453882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113e65-5b71-4fc8-81dc-c504b5107b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:09.455268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113e65-5b71-4fc8-81dc-c504b5107b47 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:09.455760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113e65-5b71-4fc8-81dc-c504b5107b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:09.456218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113e65-5b71-4fc8-81dc-c504b5107b47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:09.466228 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773badb9-ee8b-49a1-8bda-825e6edacac3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:09.488153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c113e65-5b71-4fc8-81dc-c504b5107b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:09.490606 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c113e65-5b71-4fc8-81dc-c504b5107b47 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:09.521177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df82f41-1aa2-4066-84c3-400b84950942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-8c113e65-5b71-4fc8-81dc-c504b5107b47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:29:09.521563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:09.522493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:29:09.523315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:29:09.523315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:29:09.523315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:09.523486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df82f41-1aa2-4066-84c3-400b84950942 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:09.523486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df82f41-1aa2-4066-84c3-400b84950942 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:09.523486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df82f41-1aa2-4066-84c3-400b84950942 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:09.523659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 9/18] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:29:09.524778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df82f41-1aa2-4066-84c3-400b84950942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:09.526659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df82f41-1aa2-4066-84c3-400b84950942 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:09.528505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 10/19] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:09.530505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113e65-5b71-4fc8-81dc-c504b5107b47 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:09.535325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df82f41-1aa2-4066-84c3-400b84950942 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:09.594366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:09.626898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:09.648810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113e65-5b71-4fc8-81dc-c504b5107b47 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:09.650001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df82f41-1aa2-4066-84c3-400b84950942 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:09.686843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77525e1e-fa7b-4150-9cf5-54697bc46a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:09.687448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77525e1e-fa7b-4150-9cf5-54697bc46a02 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:09.687724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77525e1e-fa7b-4150-9cf5-54697bc46a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:09.687988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77525e1e-fa7b-4150-9cf5-54697bc46a02 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:09.688622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77525e1e-fa7b-4150-9cf5-54697bc46a02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:09.688922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77525e1e-fa7b-4150-9cf5-54697bc46a02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:09.689430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f0398-1401-4900-9caf-3c0d5d5f8acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:09.690043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f0398-1401-4900-9caf-3c0d5d5f8acc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:09.690300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77525e1e-fa7b-4150-9cf5-54697bc46a02 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:09.690387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f0398-1401-4900-9caf-3c0d5d5f8acc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:09.690612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f0398-1401-4900-9caf-3c0d5d5f8acc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:09.691340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f0398-1401-4900-9caf-3c0d5d5f8acc None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.domain.id=default, user.id=557208629c7f4826b594ef28cabef8ea) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:09.691646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f0398-1401-4900-9caf-3c0d5d5f8acc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:09.693115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f0398-1401-4900-9caf-3c0d5d5f8acc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:09.700062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 11/20] 10.222.0.58 () {56 vars in 1322 bytes} [Mon Aug 29 23:29:09 2022] GET /identity/v3/role_assignments?user.id=557208629c7f4826b594ef28cabef8ea&scope.domain.id=default&role.id=d86dba8efe20423abdde06031690beb8 => generated 233 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:09.709628 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 10/21] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:12.225071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6606000-1176-42a9-8b36-ec2d3f932d6f None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:12.225969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6606000-1176-42a9-8b36-ec2d3f932d6f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:12.226460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6606000-1176-42a9-8b36-ec2d3f932d6f None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:12.228405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 12/22] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:12.235305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01994c-4db5-4df9-9a63-ebbfeba8bfbf None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:12.235794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01994c-4db5-4df9-9a63-ebbfeba8bfbf None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:12.235893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01994c-4db5-4df9-9a63-ebbfeba8bfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:12.264329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-2d01994c-4db5-4df9-9a63-ebbfeba8bfbf None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:12.265540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2d01994c-4db5-4df9-9a63-ebbfeba8bfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:12.354867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 11/23] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:12.564516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de9e40-85ed-4ffa-9a6d-609c69ca641d None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:12.564708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de9e40-85ed-4ffa-9a6d-609c69ca641d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:12.565150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de9e40-85ed-4ffa-9a6d-609c69ca641d None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:12.566987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 13/24] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:12.573694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c86f1-b3c7-4323-9b3d-d30cebd6a9f8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:12.573973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c86f1-b3c7-4323-9b3d-d30cebd6a9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:12.574247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c86f1-b3c7-4323-9b3d-d30cebd6a9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:12.603220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-e93c86f1-b3c7-4323-9b3d-d30cebd6a9f8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:12.604427 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e93c86f1-b3c7-4323-9b3d-d30cebd6a9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:12.664542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:12.694218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 12/25] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:12.812582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de9e40-85ed-4ffa-9a6d-609c69ca641d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:12.943425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c9328-d997-413b-bee6-6f8c6680d170 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:12.943794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c9328-d997-413b-bee6-6f8c6680d170 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:12.943961 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c9328-d997-413b-bee6-6f8c6680d170 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:12.945684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 13/26] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:12.952285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a765653-2c86-46f3-848e-ffd5a721054b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:12.952564 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a765653-2c86-46f3-848e-ffd5a721054b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:12.952838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a765653-2c86-46f3-848e-ffd5a721054b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:12.962311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de9e40-85ed-4ffa-9a6d-609c69ca641d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:12.983274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7a765653-2c86-46f3-848e-ffd5a721054b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:12.984485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7a765653-2c86-46f3-848e-ffd5a721054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:13.048655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f45e48-0b02-46b8-a300-845857b67f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:13.049300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f45e48-0b02-46b8-a300-845857b67f1e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:13.049576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f45e48-0b02-46b8-a300-845857b67f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:13.049899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f45e48-0b02-46b8-a300-845857b67f1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:13.072334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f45e48-0b02-46b8-a300-845857b67f1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-47f45e48-0b02-46b8-a300-845857b67f1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:13.083697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:13.084618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:13.085387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:13.088358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 14/27] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:29:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:13.094502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f45e48-0b02-46b8-a300-845857b67f1e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:13.102699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:13.163484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 14/28] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:13.168435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a765653-2c86-46f3-848e-ffd5a721054b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:13.352787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a765653-2c86-46f3-848e-ffd5a721054b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:13.365638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f45e48-0b02-46b8-a300-845857b67f1e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:13.366671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:13.416127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0936f-2306-4d14-b3a4-05408cd1f95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:13.416746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0936f-2306-4d14-b3a4-05408cd1f95e None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:13.417075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0936f-2306-4d14-b3a4-05408cd1f95e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:13.417350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0936f-2306-4d14-b3a4-05408cd1f95e None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:13.418067 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0936f-2306-4d14-b3a4-05408cd1f95e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:13.420333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0936f-2306-4d14-b3a4-05408cd1f95e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:13.437802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54354208-c9a4-4d2e-9cc2-34ad2c91a54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:13.442294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54354208-c9a4-4d2e-9cc2-34ad2c91a54e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:13.442294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54354208-c9a4-4d2e-9cc2-34ad2c91a54e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:13.442294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54354208-c9a4-4d2e-9cc2-34ad2c91a54e None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:13.442294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54354208-c9a4-4d2e-9cc2-34ad2c91a54e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:13.442294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54354208-c9a4-4d2e-9cc2-34ad2c91a54e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:13.442294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54354208-c9a4-4d2e-9cc2-34ad2c91a54e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:13.451989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 15/29] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:13.453456 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b38d6d-7c0b-49aa-9a8d-3a239aacd1de None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:13.453743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b38d6d-7c0b-49aa-9a8d-3a239aacd1de None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:13.454017 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b38d6d-7c0b-49aa-9a8d-3a239aacd1de None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:13.455758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 16/30] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:13.457002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b38d6d-7c0b-49aa-9a8d-3a239aacd1de None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:13.542093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:13.581075 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b38d6d-7c0b-49aa-9a8d-3a239aacd1de None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:13.635933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f813f82-795d-42c3-8d25-159e48afa9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:13.637064 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f813f82-795d-42c3-8d25-159e48afa9ff None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:13.637642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f813f82-795d-42c3-8d25-159e48afa9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:13.638162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f813f82-795d-42c3-8d25-159e48afa9ff None admin] PATH_INFO: `/v3/users/557208629c7f4826b594ef28cabef8ea` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:13.640422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f813f82-795d-42c3-8d25-159e48afa9ff None admin] RBAC: Authorizing `identity:get_user(user_id=557208629c7f4826b594ef28cabef8ea)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:13.643776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f813f82-795d-42c3-8d25-159e48afa9ff None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:13.646169 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 17/31] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:13 2022] GET /identity/v3/users/557208629c7f4826b594ef28cabef8ea => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:13.648579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212373d-3efe-4673-ab94-f5b14f3692f3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:13.649062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212373d-3efe-4673-ab94-f5b14f3692f3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:13.649479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212373d-3efe-4673-ab94-f5b14f3692f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:13.655201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 15/32] 10.222.0.58 () {60 vars in 1131 bytes} [Mon Aug 29 23:29:13 2022] POST /identity/v3/roles => generated 219 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:13.656299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0936f-2306-4d14-b3a4-05408cd1f95e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:13.683202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2212373d-3efe-4673-ab94-f5b14f3692f3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:13.686535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2212373d-3efe-4673-ab94-f5b14f3692f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:13.810632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 18/33] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:13.811707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212373d-3efe-4673-ab94-f5b14f3692f3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:13.895147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:13.932813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0936f-2306-4d14-b3a4-05408cd1f95e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:13.934415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212373d-3efe-4673-ab94-f5b14f3692f3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:13.983420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4435d-236f-4813-a6ff-c301314de228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:13.984079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4435d-236f-4813-a6ff-c301314de228 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:13.984360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4435d-236f-4813-a6ff-c301314de228 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:13.984638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4435d-236f-4813-a6ff-c301314de228 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:13.986093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4435d-236f-4813-a6ff-c301314de228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:13.987805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4435d-236f-4813-a6ff-c301314de228 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:13.996045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 19/34] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:13.997219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4435d-236f-4813-a6ff-c301314de228 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:14.017987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f764e4-2fbe-4825-a6a5-9871786ccdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:14.019579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f764e4-2fbe-4825-a6a5-9871786ccdba None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:14.020444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f764e4-2fbe-4825-a6a5-9871786ccdba None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:14.021336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f764e4-2fbe-4825-a6a5-9871786ccdba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:14.040137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f764e4-2fbe-4825-a6a5-9871786ccdba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:14.042200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f764e4-2fbe-4825-a6a5-9871786ccdba None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-e4f764e4-2fbe-4825-a6a5-9871786ccdba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:29:14.080354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:14.081914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:29:14.083455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:29:14.083455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:29:14.083455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:14.086823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 16/35] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:14.097089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1f1a2-85d9-4bf2-8712-df5b9dac451f None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:14.097932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1f1a2-85d9-4bf2-8712-df5b9dac451f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:14.098682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1f1a2-85d9-4bf2-8712-df5b9dac451f None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:14.101784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 17/36] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:14.104202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1f1a2-85d9-4bf2-8712-df5b9dac451f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:14.133292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4435d-236f-4813-a6ff-c301314de228 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:14.142076 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:14.177885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:14.194749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821c13f-db1a-42ad-9ee3-9ba0d8c32943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:14.195907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821c13f-db1a-42ad-9ee3-9ba0d8c32943 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:14.196475 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821c13f-db1a-42ad-9ee3-9ba0d8c32943 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:14.196968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821c13f-db1a-42ad-9ee3-9ba0d8c32943 None admin] PATH_INFO: `/v3/domains/default/users/557208629c7f4826b594ef28cabef8ea/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:14.206158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821c13f-db1a-42ad-9ee3-9ba0d8c32943 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=557208629c7f4826b594ef28cabef8ea, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:14.209003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821c13f-db1a-42ad-9ee3-9ba0d8c32943 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:14.312469 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1f1a2-85d9-4bf2-8712-df5b9dac451f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:14.377514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44753e7-0103-4bba-90ed-3752de07dcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:14.378629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44753e7-0103-4bba-90ed-3752de07dcee None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:14.379149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44753e7-0103-4bba-90ed-3752de07dcee None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:14.379663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44753e7-0103-4bba-90ed-3752de07dcee None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:14.380901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44753e7-0103-4bba-90ed-3752de07dcee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:14.381463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44753e7-0103-4bba-90ed-3752de07dcee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:14.384236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44753e7-0103-4bba-90ed-3752de07dcee None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:14.413116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 18/37] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:14.413116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f0d88-a6da-41cf-a0a2-143ac9a364bf None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:14.413116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f0d88-a6da-41cf-a0a2-143ac9a364bf None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:14.413116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f0d88-a6da-41cf-a0a2-143ac9a364bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:14.431971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 20/38] 10.222.0.58 () {58 vars in 1361 bytes} [Mon Aug 29 23:29:13 2022] PUT /identity/v3/domains/default/users/557208629c7f4826b594ef28cabef8ea/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:29:14.433121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821c13f-db1a-42ad-9ee3-9ba0d8c32943 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:14.451080 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-3e0f0d88-a6da-41cf-a0a2-143ac9a364bf None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:14.454915 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3e0f0d88-a6da-41cf-a0a2-143ac9a364bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:14.557812 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 19/39] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:14.567508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f0d88-a6da-41cf-a0a2-143ac9a364bf None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:14.578388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821c13f-db1a-42ad-9ee3-9ba0d8c32943 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:14.584744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:14.623263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b57e4-ebfd-4894-b982-84ee5ca4c3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:14.625144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b57e4-ebfd-4894-b982-84ee5ca4c3b4 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:14.625599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b57e4-ebfd-4894-b982-84ee5ca4c3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:14.626024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b57e4-ebfd-4894-b982-84ee5ca4c3b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:14.626969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b57e4-ebfd-4894-b982-84ee5ca4c3b4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:14.628902 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b57e4-ebfd-4894-b982-84ee5ca4c3b4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:14.713974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f0d88-a6da-41cf-a0a2-143ac9a364bf None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:14.745053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 21/40] 10.222.0.58 () {60 vars in 1131 bytes} [Mon Aug 29 23:29:14 2022] POST /identity/v3/roles => generated 225 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:14.753253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bf875-afa6-438c-b88f-2cf373dd1169 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:14.754892 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bf875-afa6-438c-b88f-2cf373dd1169 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:14.755188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bf875-afa6-438c-b88f-2cf373dd1169 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:14.758188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 22/41] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:14.767324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee834cc-2236-4bad-a4e8-126aa482328d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:14.767633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee834cc-2236-4bad-a4e8-126aa482328d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:14.767884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee834cc-2236-4bad-a4e8-126aa482328d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:14.783798 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1eba80-2236-40b1-9746-f7f729167ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:14.784644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eba80-2236-40b1-9746-f7f729167ad2 None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:14.784928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eba80-2236-40b1-9746-f7f729167ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:14.785210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eba80-2236-40b1-9746-f7f729167ad2 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:14.786245 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1eba80-2236-40b1-9746-f7f729167ad2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:14.791971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1eba80-2236-40b1-9746-f7f729167ad2 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:14.821093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-eee834cc-2236-4bad-a4e8-126aa482328d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:14.823332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-eee834cc-2236-4bad-a4e8-126aa482328d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:14.906460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 20/42] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:29:14 2022] POST /identity/v3/projects => generated 301 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:14.908645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1eba80-2236-40b1-9746-f7f729167ad2 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:14.913878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:14.977672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 23/43] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:29:15.018198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:15.080114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1eba80-2236-40b1-9746-f7f729167ad2 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:15.148660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6e8ae-dd3b-46ae-a7f3-4231c4eece1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:15.149997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6e8ae-dd3b-46ae-a7f3-4231c4eece1c None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:15.150284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6e8ae-dd3b-46ae-a7f3-4231c4eece1c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:15.150579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6e8ae-dd3b-46ae-a7f3-4231c4eece1c None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:15.151278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6e8ae-dd3b-46ae-a7f3-4231c4eece1c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:15.153260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6e8ae-dd3b-46ae-a7f3-4231c4eece1c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:15.239300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 21/44] 10.222.0.58 () {60 vars in 1131 bytes} [Mon Aug 29 23:29:14 2022] POST /identity/v3/roles => generated 223 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:15.316888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee834cc-2236-4bad-a4e8-126aa482328d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:15.471760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee834cc-2236-4bad-a4e8-126aa482328d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:15.522335 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2982f9-3e4b-44ed-a2ec-08ad838693b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:15.523623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2982f9-3e4b-44ed-a2ec-08ad838693b9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:15.523913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2982f9-3e4b-44ed-a2ec-08ad838693b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:15.524197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2982f9-3e4b-44ed-a2ec-08ad838693b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:15.525687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2982f9-3e4b-44ed-a2ec-08ad838693b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:15.527251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2982f9-3e4b-44ed-a2ec-08ad838693b9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:15.529419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 24/45] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:15.535196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6e8ae-dd3b-46ae-a7f3-4231c4eece1c None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:15.685677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6e8ae-dd3b-46ae-a7f3-4231c4eece1c None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:15.709434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:15.725415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfeeb9-a8a5-4236-91d7-7f66eb51e93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:15.726182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfeeb9-a8a5-4236-91d7-7f66eb51e93b None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:15.726690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfeeb9-a8a5-4236-91d7-7f66eb51e93b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:15.727090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfeeb9-a8a5-4236-91d7-7f66eb51e93b None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:15.727877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfeeb9-a8a5-4236-91d7-7f66eb51e93b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:15.729441 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfeeb9-a8a5-4236-91d7-7f66eb51e93b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:15.838368 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 22/46] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:29:15 2022] POST /identity/v3/projects => generated 312 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:16.657631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162999f9-bece-490a-afb6-e2cb8adaf3ed None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:16.658118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162999f9-bece-490a-afb6-e2cb8adaf3ed None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:16.658118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162999f9-bece-490a-afb6-e2cb8adaf3ed None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:16.659492 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 25/47] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:16.664758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4cedb-eea3-4f19-93c3-4a1546017c1f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:16.665194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4cedb-eea3-4f19-93c3-4a1546017c1f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:16.665385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4cedb-eea3-4f19-93c3-4a1546017c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:16.686269 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f1e4cedb-eea3-4f19-93c3-4a1546017c1f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:16.687523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f1e4cedb-eea3-4f19-93c3-4a1546017c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:16.764477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 23/48] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:17.034541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162999f9-bece-490a-afb6-e2cb8adaf3ed None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:17.206444 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:17.261665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162999f9-bece-490a-afb6-e2cb8adaf3ed None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:17.296569 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81efe31-3a82-41c3-ac21-00a4849fd047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:17.297187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81efe31-3a82-41c3-ac21-00a4849fd047 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:17.297467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81efe31-3a82-41c3-ac21-00a4849fd047 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:17.297744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81efe31-3a82-41c3-ac21-00a4849fd047 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:17.309900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81efe31-3a82-41c3-ac21-00a4849fd047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c81efe31-3a82-41c3-ac21-00a4849fd047 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:17.314885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:17.315775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:17.316542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 26/49] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:29:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:17.323047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4cedb-eea3-4f19-93c3-4a1546017c1f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:17.462185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4cedb-eea3-4f19-93c3-4a1546017c1f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:17.465490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:17.522977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd741058-bf6d-4454-bdf2-8dfaab698ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:17.524040 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd741058-bf6d-4454-bdf2-8dfaab698ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:17.524807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd741058-bf6d-4454-bdf2-8dfaab698ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:17.525306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd741058-bf6d-4454-bdf2-8dfaab698ab0 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:17.526601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd741058-bf6d-4454-bdf2-8dfaab698ab0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:17.527151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd741058-bf6d-4454-bdf2-8dfaab698ab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:17.530442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd741058-bf6d-4454-bdf2-8dfaab698ab0 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:17.553457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 24/50] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:17.559144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81efe31-3a82-41c3-ac21-00a4849fd047 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:17.661958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81efe31-3a82-41c3-ac21-00a4849fd047 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:17.673246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:17.696316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e07525-dc22-4f4c-96b5-12994598664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:17.696955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e07525-dc22-4f4c-96b5-12994598664d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:17.697236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e07525-dc22-4f4c-96b5-12994598664d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:17.697511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e07525-dc22-4f4c-96b5-12994598664d None admin] PATH_INFO: `/v3/users/557208629c7f4826b594ef28cabef8ea` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:17.698716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e07525-dc22-4f4c-96b5-12994598664d None admin] RBAC: Authorizing `identity:get_user(user_id=557208629c7f4826b594ef28cabef8ea)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:17.700850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e07525-dc22-4f4c-96b5-12994598664d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:17.702487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 27/51] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:17 2022] GET /identity/v3/users/557208629c7f4826b594ef28cabef8ea => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:17.709553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd741058-bf6d-4454-bdf2-8dfaab698ab0 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:17.849763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aaefb-0fc3-409a-b28b-13e72b8c121e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:17.850047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aaefb-0fc3-409a-b28b-13e72b8c121e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:17.850444 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aaefb-0fc3-409a-b28b-13e72b8c121e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:17.852307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 28/52] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:17.861728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7def89-2b19-4bcb-9e88-8049bb0a981a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:17.862013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7def89-2b19-4bcb-9e88-8049bb0a981a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:17.862295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7def89-2b19-4bcb-9e88-8049bb0a981a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:17.883455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-cf7def89-2b19-4bcb-9e88-8049bb0a981a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:17.884692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cf7def89-2b19-4bcb-9e88-8049bb0a981a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:17.934549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd741058-bf6d-4454-bdf2-8dfaab698ab0 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:17.972409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1adac-a559-4d15-9168-a4d1aab0f70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:17.972873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 29/53] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:17.973114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1adac-a559-4d15-9168-a4d1aab0f70b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:17.973349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1adac-a559-4d15-9168-a4d1aab0f70b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:17.973626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1adac-a559-4d15-9168-a4d1aab0f70b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:17.974937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1adac-a559-4d15-9168-a4d1aab0f70b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:17.976473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1adac-a559-4d15-9168-a4d1aab0f70b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:17.978209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 25/54] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:17.985452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7def89-2b19-4bcb-9e88-8049bb0a981a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:18.102842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7def89-2b19-4bcb-9e88-8049bb0a981a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:18.114040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:18.142680 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f8c2c-384b-48c8-b775-5fd861298073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:18.143299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f8c2c-384b-48c8-b775-5fd861298073 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:18.143577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f8c2c-384b-48c8-b775-5fd861298073 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:18.143848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f8c2c-384b-48c8-b775-5fd861298073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:18.144578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f8c2c-384b-48c8-b775-5fd861298073 None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.domain.id=default, user.id=557208629c7f4826b594ef28cabef8ea) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:18.144892 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f8c2c-384b-48c8-b775-5fd861298073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:18.146364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f8c2c-384b-48c8-b775-5fd861298073 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:18.153211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 30/55] 10.222.0.58 () {56 vars in 1322 bytes} [Mon Aug 29 23:29:17 2022] GET /identity/v3/role_assignments?user.id=557208629c7f4826b594ef28cabef8ea&scope.domain.id=default&role.id=d86dba8efe20423abdde06031690beb8 => generated 530 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:18.251258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1adac-a559-4d15-9168-a4d1aab0f70b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:18.290179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:18.381000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1adac-a559-4d15-9168-a4d1aab0f70b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:18.416831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4b0dd-db8f-44e8-a6cb-94678f148af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:18.417472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4b0dd-db8f-44e8-a6cb-94678f148af2 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:18.417756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4b0dd-db8f-44e8-a6cb-94678f148af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:18.418249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4b0dd-db8f-44e8-a6cb-94678f148af2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:18.419347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4b0dd-db8f-44e8-a6cb-94678f148af2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:18.420826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4b0dd-db8f-44e8-a6cb-94678f148af2 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:18.422405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 26/56] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:18.428366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f8c2c-384b-48c8-b775-5fd861298073 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:18.535881 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f8c2c-384b-48c8-b775-5fd861298073 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:18.571548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bfbb6-5c23-4003-8cbd-dbc9a908c33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:18.572157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bfbb6-5c23-4003-8cbd-dbc9a908c33d None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:18.572436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bfbb6-5c23-4003-8cbd-dbc9a908c33d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:18.572707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bfbb6-5c23-4003-8cbd-dbc9a908c33d None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:18.573350 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bfbb6-5c23-4003-8cbd-dbc9a908c33d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:18.575130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bfbb6-5c23-4003-8cbd-dbc9a908c33d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:18.655442 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 31/57] 10.222.0.58 () {60 vars in 1140 bytes} [Mon Aug 29 23:29:18 2022] POST /identity/v3/projects => generated 298 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:20.644568 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62264c15-eedc-4d7e-a901-f64e144fba00 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:20.645060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62264c15-eedc-4d7e-a901-f64e144fba00 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:20.645060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62264c15-eedc-4d7e-a901-f64e144fba00 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:20.646361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 27/58] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:20.652678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7f29-2449-498c-91c7-caa64921ed85 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:20.653082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7f29-2449-498c-91c7-caa64921ed85 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:20.653400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7f29-2449-498c-91c7-caa64921ed85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:20.673076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f46e7f29-2449-498c-91c7-caa64921ed85 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:20.674312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f46e7f29-2449-498c-91c7-caa64921ed85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:20.692425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:20.748207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 32/59] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:21.023050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62264c15-eedc-4d7e-a901-f64e144fba00 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:21.131707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:21.136092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62264c15-eedc-4d7e-a901-f64e144fba00 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:21.174393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ef068-e5c8-40cf-b6ff-a26a5eb5fcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:21.175063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ef068-e5c8-40cf-b6ff-a26a5eb5fcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:21.175344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ef068-e5c8-40cf-b6ff-a26a5eb5fcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:21.175619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ef068-e5c8-40cf-b6ff-a26a5eb5fcb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:21.176795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ef068-e5c8-40cf-b6ff-a26a5eb5fcb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:21.178256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ef068-e5c8-40cf-b6ff-a26a5eb5fcb3 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:21.179876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 28/60] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:21.185846 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e7f29-2449-498c-91c7-caa64921ed85 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:21.295603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e7f29-2449-498c-91c7-caa64921ed85 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:21.330850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef0000-b711-4e76-94a8-3552ac7d7bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:21.331461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef0000-b711-4e76-94a8-3552ac7d7bb4 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:21.331739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef0000-b711-4e76-94a8-3552ac7d7bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:21.332013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef0000-b711-4e76-94a8-3552ac7d7bb4 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:21.332677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef0000-b711-4e76-94a8-3552ac7d7bb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:21.334165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef0000-b711-4e76-94a8-3552ac7d7bb4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:21.425384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 33/61] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:29:21 2022] POST /identity/v3/users => generated 275 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:24.458875 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1b2e4-448d-4a59-ad62-c7ad4ced611c None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:24.459924 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1b2e4-448d-4a59-ad62-c7ad4ced611c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:24.460417 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1b2e4-448d-4a59-ad62-c7ad4ced611c None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:24.462044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 29/62] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:29:24.469339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe410d-d234-4024-a75c-65b4fc201f13 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:24.469956 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe410d-d234-4024-a75c-65b4fc201f13 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:24.470359 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe410d-d234-4024-a75c-65b4fc201f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:24.501580 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-27fe410d-d234-4024-a75c-65b4fc201f13 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:24.503097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-27fe410d-d234-4024-a75c-65b4fc201f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:24.512417 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:24.604900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 34/63] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:24.817661 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d652b96-e3ad-4fd7-a7c9-2b510f4394a7 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:24.818206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d652b96-e3ad-4fd7-a7c9-2b510f4394a7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:24.818668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d652b96-e3ad-4fd7-a7c9-2b510f4394a7 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:24.820184 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 30/64] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:24.826107 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059cb2-1fa7-421a-93e0-293884616370 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:24.826562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059cb2-1fa7-421a-93e0-293884616370 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:24.826914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059cb2-1fa7-421a-93e0-293884616370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:24.856612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-39059cb2-1fa7-421a-93e0-293884616370 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:24.857633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-39059cb2-1fa7-421a-93e0-293884616370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:24.909458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b50070-1b40-4bd6-8574-4cd9701677e6 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:24.910235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b50070-1b40-4bd6-8574-4cd9701677e6 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:24.910944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b50070-1b40-4bd6-8574-4cd9701677e6 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:24.913135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 31/65] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:29:24.919345 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba55d30-b03b-43c1-8d5f-b574f21005ba None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:24.919660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba55d30-b03b-43c1-8d5f-b574f21005ba None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:24.919898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba55d30-b03b-43c1-8d5f-b574f21005ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:24.959044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-aba55d30-b03b-43c1-8d5f-b574f21005ba None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:24.960263 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-aba55d30-b03b-43c1-8d5f-b574f21005ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:24.967077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 35/66] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:25.015389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39059cb2-1fa7-421a-93e0-293884616370 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:25.072126 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 32/67] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:25.125202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:25.177348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39059cb2-1fa7-421a-93e0-293884616370 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:25.217412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a588051-ebae-4eb7-8087-a7c84d837a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:25.218075 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a588051-ebae-4eb7-8087-a7c84d837a28 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:25.218354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a588051-ebae-4eb7-8087-a7c84d837a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:25.218646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a588051-ebae-4eb7-8087-a7c84d837a28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:25.229885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a588051-ebae-4eb7-8087-a7c84d837a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1a588051-ebae-4eb7-8087-a7c84d837a28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:25.235703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:25.236575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:25.237744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 36/68] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:25.242845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba55d30-b03b-43c1-8d5f-b574f21005ba None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:25.328229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a588051-ebae-4eb7-8087-a7c84d837a28 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:25.373987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:25.389034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba55d30-b03b-43c1-8d5f-b574f21005ba None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:25.424028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8299f4e-a286-4555-968b-3980774fa866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:25.425239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8299f4e-a286-4555-968b-3980774fa866 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:25.425849 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8299f4e-a286-4555-968b-3980774fa866 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:25.426308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8299f4e-a286-4555-968b-3980774fa866 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:25.427203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8299f4e-a286-4555-968b-3980774fa866 None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:25.427693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8299f4e-a286-4555-968b-3980774fa866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:25.429456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8299f4e-a286-4555-968b-3980774fa866 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:25.440796 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 33/69] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:25.442562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec81488-8796-4129-a68b-8a270bdd41a2 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:25.443050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec81488-8796-4129-a68b-8a270bdd41a2 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:25.443512 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec81488-8796-4129-a68b-8a270bdd41a2 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:25.449036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 34/70] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:25.449036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec81488-8796-4129-a68b-8a270bdd41a2 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:25.475211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a588051-ebae-4eb7-8087-a7c84d837a28 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:25.506820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:25.512774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1dfd01-c1bb-49d9-b1d0-20a7a659c11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:25.513446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1dfd01-c1bb-49d9-b1d0-20a7a659c11e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:25.519424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1dfd01-c1bb-49d9-b1d0-20a7a659c11e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:25.519424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1dfd01-c1bb-49d9-b1d0-20a7a659c11e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:25.524670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1dfd01-c1bb-49d9-b1d0-20a7a659c11e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-4a1dfd01-c1bb-49d9-b1d0-20a7a659c11e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:25.530173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:25.531093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:25.531852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 37/71] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:29:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:29:25.532490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1dfd01-c1bb-49d9-b1d0-20a7a659c11e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:25.560208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec81488-8796-4129-a68b-8a270bdd41a2 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:25.598373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd8c90-cfc8-4a55-961b-70176f514a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:25.599072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd8c90-cfc8-4a55-961b-70176f514a3c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:25.599355 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd8c90-cfc8-4a55-961b-70176f514a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:25.599631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd8c90-cfc8-4a55-961b-70176f514a3c None admin] PATH_INFO: `/v3/users/3d90934d388e445184cf04273effcac0` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:25.616463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd8c90-cfc8-4a55-961b-70176f514a3c None admin] RBAC: Authorizing `identity:get_user(user_id=3d90934d388e445184cf04273effcac0)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:25.617954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd8c90-cfc8-4a55-961b-70176f514a3c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:25.619682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 35/72] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:25 2022] GET /identity/v3/users/3d90934d388e445184cf04273effcac0 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:25.625347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d941d-e2e8-48f4-a80a-75018eb4205a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:25.625347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d941d-e2e8-48f4-a80a-75018eb4205a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:25.625347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d941d-e2e8-48f4-a80a-75018eb4205a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:25.642406 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-681d941d-e2e8-48f4-a80a-75018eb4205a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:25.643569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-681d941d-e2e8-48f4-a80a-75018eb4205a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:25.647515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1dfd01-c1bb-49d9-b1d0-20a7a659c11e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:25.659959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:25.683300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1683fa-b9ee-49c1-9793-c263c0993d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:25.684101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1683fa-b9ee-49c1-9793-c263c0993d6c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:25.684379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1683fa-b9ee-49c1-9793-c263c0993d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:25.684651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1683fa-b9ee-49c1-9793-c263c0993d6c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:25.695194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1683fa-b9ee-49c1-9793-c263c0993d6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-7e1683fa-b9ee-49c1-9793-c263c0993d6c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:25.700111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:25.701175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:25.702093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 38/73] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:29:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:25.702788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e449add-a930-43f5-8059-ba7f13fe6d3b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:25.702851 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e449add-a930-43f5-8059-ba7f13fe6d3b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:25.703236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e449add-a930-43f5-8059-ba7f13fe6d3b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:25.704642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 39/74] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:25.705269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e449add-a930-43f5-8059-ba7f13fe6d3b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:25.723672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 36/75] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:25.724563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d941d-e2e8-48f4-a80a-75018eb4205a None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:25.776784 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:25.829372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d941d-e2e8-48f4-a80a-75018eb4205a None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:25.836110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e449add-a930-43f5-8059-ba7f13fe6d3b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:25.865959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2207d-d651-467b-877c-6416c7dc1263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:25.866598 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2207d-d651-467b-877c-6416c7dc1263 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:25.866885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2207d-d651-467b-877c-6416c7dc1263 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:25.867159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2207d-d651-467b-877c-6416c7dc1263 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:25.873640 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd49727-93c8-4215-bbb6-f1b6fd56b384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:25.874263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd49727-93c8-4215-bbb6-f1b6fd56b384 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:25.874577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd49727-93c8-4215-bbb6-f1b6fd56b384 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:25.874856 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd49727-93c8-4215-bbb6-f1b6fd56b384 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:25.875943 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd49727-93c8-4215-bbb6-f1b6fd56b384 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:25.876256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd49727-93c8-4215-bbb6-f1b6fd56b384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:25.878051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd49727-93c8-4215-bbb6-f1b6fd56b384 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:25.884985 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:25.892396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 40/76] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:25.892396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd49727-93c8-4215-bbb6-f1b6fd56b384 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:25.940504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2207d-d651-467b-877c-6416c7dc1263 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:25.941894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2207d-d651-467b-877c-6416c7dc1263 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:26.004087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd49727-93c8-4215-bbb6-f1b6fd56b384 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:26.017850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:26.039745 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99794f-1749-4ca3-afda-98eef660c065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:26.040360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99794f-1749-4ca3-afda-98eef660c065 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:26.040639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99794f-1749-4ca3-afda-98eef660c065 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:26.040911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99794f-1749-4ca3-afda-98eef660c065 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:26.041578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99794f-1749-4ca3-afda-98eef660c065 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:26.041900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99794f-1749-4ca3-afda-98eef660c065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:26.044074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99794f-1749-4ca3-afda-98eef660c065 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:26.046550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 37/77] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:26.047816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d33b2-db6e-4a0e-8202-8fb289e4e1a2 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:26.048107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d33b2-db6e-4a0e-8202-8fb289e4e1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:26.048382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d33b2-db6e-4a0e-8202-8fb289e4e1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:26.053655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 41/78] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:29:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:26.054556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99794f-1749-4ca3-afda-98eef660c065 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:26.067735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f25d33b2-db6e-4a0e-8202-8fb289e4e1a2 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:26.068962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f25d33b2-db6e-4a0e-8202-8fb289e4e1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:26.117695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:26.141223 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 38/79] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:26.142308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d33b2-db6e-4a0e-8202-8fb289e4e1a2 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:26.174993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99794f-1749-4ca3-afda-98eef660c065 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:26.211144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752f8d7-a4e5-4812-a3a0-d39d7f2941ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:26.211704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752f8d7-a4e5-4812-a3a0-d39d7f2941ba None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:26.211967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752f8d7-a4e5-4812-a3a0-d39d7f2941ba None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:26.212285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752f8d7-a4e5-4812-a3a0-d39d7f2941ba None admin] PATH_INFO: `/v3/users/557208629c7f4826b594ef28cabef8ea` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:26.213392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752f8d7-a4e5-4812-a3a0-d39d7f2941ba None admin] RBAC: Authorizing `identity:get_user(user_id=557208629c7f4826b594ef28cabef8ea)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:26.215829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752f8d7-a4e5-4812-a3a0-d39d7f2941ba None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:26.222118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 42/80] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:26 2022] GET /identity/v3/users/557208629c7f4826b594ef28cabef8ea => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:26.222118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752f8d7-a4e5-4812-a3a0-d39d7f2941ba None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:26.262749 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d33b2-db6e-4a0e-8202-8fb289e4e1a2 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:26.268322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:26.300944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef544d-1e12-4939-8e71-d0d36490199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:26.301599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef544d-1e12-4939-8e71-d0d36490199f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:26.301889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef544d-1e12-4939-8e71-d0d36490199f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:26.302170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef544d-1e12-4939-8e71-d0d36490199f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:26.317766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef544d-1e12-4939-8e71-d0d36490199f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-94ef544d-1e12-4939-8e71-d0d36490199f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:26.325916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:26.327164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:26.328093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:26.329253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 39/81] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:26.330064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef544d-1e12-4939-8e71-d0d36490199f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:26.338393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752f8d7-a4e5-4812-a3a0-d39d7f2941ba None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:26.376314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b50016-29ee-453a-8e34-ed64bcc18ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:26.376952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50016-29ee-453a-8e34-ed64bcc18ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:26.377228 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50016-29ee-453a-8e34-ed64bcc18ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:26.377498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50016-29ee-453a-8e34-ed64bcc18ba8 None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:26.378728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:26.378837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b50016-29ee-453a-8e34-ed64bcc18ba8 None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:26.380264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b50016-29ee-453a-8e34-ed64bcc18ba8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:26.381918 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 43/82] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:26 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:26.387634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b50016-29ee-453a-8e34-ed64bcc18ba8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:26.460208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef544d-1e12-4939-8e71-d0d36490199f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:26.489420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b50016-29ee-453a-8e34-ed64bcc18ba8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:26.494785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea53d21-3635-4940-9abf-7dac20360707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:26.496326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea53d21-3635-4940-9abf-7dac20360707 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:26.496850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea53d21-3635-4940-9abf-7dac20360707 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:26.497168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea53d21-3635-4940-9abf-7dac20360707 None admin] PATH_INFO: `/v3/users/3d90934d388e445184cf04273effcac0` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:26.498364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea53d21-3635-4940-9abf-7dac20360707 None admin] RBAC: Authorizing `identity:get_user(user_id=3d90934d388e445184cf04273effcac0)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:26.499871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea53d21-3635-4940-9abf-7dac20360707 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:26.501382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:26.501490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 40/83] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:26 2022] GET /identity/v3/users/3d90934d388e445184cf04273effcac0 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:26.502215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea53d21-3635-4940-9abf-7dac20360707 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:26.530585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be67686-d509-4994-bd3b-42eb80d4733f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:26.531194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be67686-d509-4994-bd3b-42eb80d4733f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:26.531472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be67686-d509-4994-bd3b-42eb80d4733f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:26.532024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be67686-d509-4994-bd3b-42eb80d4733f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:26.603238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be67686-d509-4994-bd3b-42eb80d4733f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:26.605377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be67686-d509-4994-bd3b-42eb80d4733f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:26.607106 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea53d21-3635-4940-9abf-7dac20360707 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:26.614895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:26.653785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d316d6-17ff-4977-8dc1-4f9fce8e780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:26.654401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d316d6-17ff-4977-8dc1-4f9fce8e780f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:26.654715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d316d6-17ff-4977-8dc1-4f9fce8e780f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:26.654997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d316d6-17ff-4977-8dc1-4f9fce8e780f None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:26.655675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d316d6-17ff-4977-8dc1-4f9fce8e780f None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:26.655987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d316d6-17ff-4977-8dc1-4f9fce8e780f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:26.657475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d316d6-17ff-4977-8dc1-4f9fce8e780f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:26.670537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 41/84] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:26.670537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d316d6-17ff-4977-8dc1-4f9fce8e780f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:26.715459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 44/85] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:26.716347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be67686-d509-4994-bd3b-42eb80d4733f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:26.730477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:26.794305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d316d6-17ff-4977-8dc1-4f9fce8e780f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:26.830636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5162a5a-5f38-49f6-8df0-becb95e034da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:26.831246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5162a5a-5f38-49f6-8df0-becb95e034da None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:26.831529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5162a5a-5f38-49f6-8df0-becb95e034da None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:26.831802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5162a5a-5f38-49f6-8df0-becb95e034da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:26.832534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5162a5a-5f38-49f6-8df0-becb95e034da None admin] RBAC: Adding query filter params (role.id=4d98a2b16f8f42bca3e5dac0374cdb54, scope.project.id=290b724bdca948a29018c8ecc35d1c8b, user.id=3d90934d388e445184cf04273effcac0) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:26.832848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5162a5a-5f38-49f6-8df0-becb95e034da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:26.833132 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be67686-d509-4994-bd3b-42eb80d4733f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:26.834308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5162a5a-5f38-49f6-8df0-becb95e034da None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:26.843388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 42/86] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:26 2022] GET /identity/v3/role_assignments?user.id=3d90934d388e445184cf04273effcac0&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:26.844015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5162a5a-5f38-49f6-8df0-becb95e034da None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:26.872728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88cccb-1295-4fc7-9385-9c4b3e1151c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:26.873496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88cccb-1295-4fc7-9385-9c4b3e1151c8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:26.873785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88cccb-1295-4fc7-9385-9c4b3e1151c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:26.874057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88cccb-1295-4fc7-9385-9c4b3e1151c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:26.875318 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88cccb-1295-4fc7-9385-9c4b3e1151c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:26.877055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88cccb-1295-4fc7-9385-9c4b3e1151c8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:26.882473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 45/87] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:26.882473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88cccb-1295-4fc7-9385-9c4b3e1151c8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:26.884753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:26.974262 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5162a5a-5f38-49f6-8df0-becb95e034da None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:26.977034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:27.002650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88cccb-1295-4fc7-9385-9c4b3e1151c8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:27.017135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338e7b8-bffa-4862-92ce-29da34b7b0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:27.017923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338e7b8-bffa-4862-92ce-29da34b7b0b7 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:27.018317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338e7b8-bffa-4862-92ce-29da34b7b0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:27.018717 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338e7b8-bffa-4862-92ce-29da34b7b0b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:27.043868 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23e4a5-8ba8-4ed8-b395-a32a532a387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:27.045333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23e4a5-8ba8-4ed8-b395-a32a532a387d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:27.045770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23e4a5-8ba8-4ed8-b395-a32a532a387d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:27.046192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23e4a5-8ba8-4ed8-b395-a32a532a387d None admin] PATH_INFO: `/v3/users/3d90934d388e445184cf04273effcac0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:27.047575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23e4a5-8ba8-4ed8-b395-a32a532a387d None admin] RBAC: Authorizing `identity:get_user(user_id=3d90934d388e445184cf04273effcac0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:27.049084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23e4a5-8ba8-4ed8-b395-a32a532a387d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:27.050916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 46/88] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:26 2022] GET /identity/v3/users/3d90934d388e445184cf04273effcac0 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:27.051543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23e4a5-8ba8-4ed8-b395-a32a532a387d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:27.109227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338e7b8-bffa-4862-92ce-29da34b7b0b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:27.111181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338e7b8-bffa-4862-92ce-29da34b7b0b7 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:27.149215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:27.163551 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23e4a5-8ba8-4ed8-b395-a32a532a387d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:27.203702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a76e7b-df6a-4384-adb0-9da33038d826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:27.204340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a76e7b-df6a-4384-adb0-9da33038d826 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:27.204617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a76e7b-df6a-4384-adb0-9da33038d826 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:27.204889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a76e7b-df6a-4384-adb0-9da33038d826 None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:27.206244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a76e7b-df6a-4384-adb0-9da33038d826 None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:27.208391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a76e7b-df6a-4384-adb0-9da33038d826 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:27.216224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 47/89] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:27 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:27.216224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a76e7b-df6a-4384-adb0-9da33038d826 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:27.228633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:27.236936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 43/90] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:27.237831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338e7b8-bffa-4862-92ce-29da34b7b0b7 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:27.352939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a76e7b-df6a-4384-adb0-9da33038d826 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:27.366553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338e7b8-bffa-4862-92ce-29da34b7b0b7 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:27.400205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bac59-cbfa-4238-bb53-91ab9567d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:27.400828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bac59-cbfa-4238-bb53-91ab9567d463 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:27.401191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bac59-cbfa-4238-bb53-91ab9567d463 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:27.401639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bac59-cbfa-4238-bb53-91ab9567d463 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:27.402329 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bac59-cbfa-4238-bb53-91ab9567d463 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:27.403158 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:27.403861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bac59-cbfa-4238-bb53-91ab9567d463 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:27.408433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420b4b2-44b6-4b30-8425-a0ec9a8d12ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:27.409136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420b4b2-44b6-4b30-8425-a0ec9a8d12ef None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:27.409407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420b4b2-44b6-4b30-8425-a0ec9a8d12ef None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:27.409635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420b4b2-44b6-4b30-8425-a0ec9a8d12ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:27.484029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420b4b2-44b6-4b30-8425-a0ec9a8d12ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:27.485515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420b4b2-44b6-4b30-8425-a0ec9a8d12ef None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:27.516403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 48/91] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:29:27 2022] POST /identity/v3/users => generated 289 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:27.516403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bac59-cbfa-4238-bb53-91ab9567d463 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:27.640570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 44/92] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:27.641532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420b4b2-44b6-4b30-8425-a0ec9a8d12ef None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:27.647402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:27.685911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:27.690049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bac59-cbfa-4238-bb53-91ab9567d463 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:27.734466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9daf4-b634-4ba7-9102-d9022cc878fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:27.735319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9daf4-b634-4ba7-9102-d9022cc878fd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:27.735757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9daf4-b634-4ba7-9102-d9022cc878fd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:27.736167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9daf4-b634-4ba7-9102-d9022cc878fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:27.737043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9daf4-b634-4ba7-9102-d9022cc878fd None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=290b724bdca948a29018c8ecc35d1c8b, user.id=557208629c7f4826b594ef28cabef8ea) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:27.737504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9daf4-b634-4ba7-9102-d9022cc878fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:27.739037 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9daf4-b634-4ba7-9102-d9022cc878fd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:27.746472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 49/93] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:27 2022] GET /identity/v3/role_assignments?user.id=557208629c7f4826b594ef28cabef8ea&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=d86dba8efe20423abdde06031690beb8 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:27.747043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9daf4-b634-4ba7-9102-d9022cc878fd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:27.779244 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420b4b2-44b6-4b30-8425-a0ec9a8d12ef None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:27.819227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3906ec-e565-484b-ab3c-87c61d7382d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:27.820235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3906ec-e565-484b-ab3c-87c61d7382d2 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:27.820741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3906ec-e565-484b-ab3c-87c61d7382d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:27.821282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3906ec-e565-484b-ab3c-87c61d7382d2 None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:27.822964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3906ec-e565-484b-ab3c-87c61d7382d2 None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:27.825006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3906ec-e565-484b-ab3c-87c61d7382d2 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:27.827133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 45/94] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:27 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:27.837756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3906ec-e565-484b-ab3c-87c61d7382d2 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:27.901399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9daf4-b634-4ba7-9102-d9022cc878fd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:27.955599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c618306-ccb5-4191-b7fa-e70d513e5d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:27.956772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c618306-ccb5-4191-b7fa-e70d513e5d97 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:27.957267 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c618306-ccb5-4191-b7fa-e70d513e5d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:27.957792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c618306-ccb5-4191-b7fa-e70d513e5d97 None admin] PATH_INFO: `/v3/projects/955d8cd8cc0f40768c3327e1ced5b79b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:27.959250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c618306-ccb5-4191-b7fa-e70d513e5d97 None admin] RBAC: Authorizing `identity:get_project(project_id=955d8cd8cc0f40768c3327e1ced5b79b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:27.961065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c618306-ccb5-4191-b7fa-e70d513e5d97 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:27.963402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 50/95] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:27 2022] GET /identity/v3/projects/955d8cd8cc0f40768c3327e1ced5b79b => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:27.971469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c618306-ccb5-4191-b7fa-e70d513e5d97 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:27.989294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:27.993101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3906ec-e565-484b-ab3c-87c61d7382d2 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:27.998805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:28.038912 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65275cd-e716-447c-9f82-acfb3e1d359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:28.040186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65275cd-e716-447c-9f82-acfb3e1d359c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:28.040683 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65275cd-e716-447c-9f82-acfb3e1d359c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:28.041147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65275cd-e716-447c-9f82-acfb3e1d359c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:28.042352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65275cd-e716-447c-9f82-acfb3e1d359c None admin] RBAC: Adding query filter params (role.id=a38face2eeae40c28edda83a4d46614f, scope.project.id=290b724bdca948a29018c8ecc35d1c8b, user.id=3d90934d388e445184cf04273effcac0) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:28.042902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65275cd-e716-447c-9f82-acfb3e1d359c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:28.044641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65275cd-e716-447c-9f82-acfb3e1d359c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:28.052923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 46/96] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:27 2022] GET /identity/v3/role_assignments?user.id=3d90934d388e445184cf04273effcac0&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=a38face2eeae40c28edda83a4d46614f => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:28.139427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c618306-ccb5-4191-b7fa-e70d513e5d97 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:28.207406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22526c3f-4c40-46c8-a8ad-57b86d608754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:28.210016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22526c3f-4c40-46c8-a8ad-57b86d608754 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:28.210016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22526c3f-4c40-46c8-a8ad-57b86d608754 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:28.210016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22526c3f-4c40-46c8-a8ad-57b86d608754 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:28.214036 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22526c3f-4c40-46c8-a8ad-57b86d608754 None admin] RBAC: Adding query filter params (role.id=4d98a2b16f8f42bca3e5dac0374cdb54, scope.project.id=955d8cd8cc0f40768c3327e1ced5b79b, user.id=3d90934d388e445184cf04273effcac0) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:28.215180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22526c3f-4c40-46c8-a8ad-57b86d608754 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:28.217334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22526c3f-4c40-46c8-a8ad-57b86d608754 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:28.228235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 51/97] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:27 2022] GET /identity/v3/role_assignments?user.id=3d90934d388e445184cf04273effcac0&scope.project.id=955d8cd8cc0f40768c3327e1ced5b79b&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:28.979499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74cc2b-07f6-46b2-97e3-9f822d32fb29 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:28.980260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74cc2b-07f6-46b2-97e3-9f822d32fb29 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:28.980260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74cc2b-07f6-46b2-97e3-9f822d32fb29 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:28.981385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 47/98] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:28.986935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbb11a-4490-47a4-b2d2-f9e272deb6df None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:28.987386 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbb11a-4490-47a4-b2d2-f9e272deb6df None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:28.987460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbb11a-4490-47a4-b2d2-f9e272deb6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:29.011451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0cfbb11a-4490-47a4-b2d2-f9e272deb6df None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:29.012680 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0cfbb11a-4490-47a4-b2d2-f9e272deb6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:29.099081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:29.107878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 52/99] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:29.502991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74cc2b-07f6-46b2-97e3-9f822d32fb29 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:29.637964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74cc2b-07f6-46b2-97e3-9f822d32fb29 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:29.675224 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfcecc-d311-4781-b2c5-3a14200782cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:29.675862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfcecc-d311-4781-b2c5-3a14200782cf None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:29.676141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfcecc-d311-4781-b2c5-3a14200782cf None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:29.676413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfcecc-d311-4781-b2c5-3a14200782cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:29.691414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfcecc-d311-4781-b2c5-3a14200782cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-0ccfcecc-d311-4781-b2c5-3a14200782cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:29.696282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:29.697122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:29.698878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 48/100] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:29.705503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbb11a-4490-47a4-b2d2-f9e272deb6df None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:29.858728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbb11a-4490-47a4-b2d2-f9e272deb6df None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:29.896420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f96ed-a661-44f9-bda4-69d36bd1b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:29.897493 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f96ed-a661-44f9-bda4-69d36bd1b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:29.897974 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f96ed-a661-44f9-bda4-69d36bd1b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:29.898722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f96ed-a661-44f9-bda4-69d36bd1b3c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:29.899605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f96ed-a661-44f9-bda4-69d36bd1b3c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:29.900112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f96ed-a661-44f9-bda4-69d36bd1b3c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:29.902371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f96ed-a661-44f9-bda4-69d36bd1b3c1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:29.912190 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 53/101] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:29.921897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfcecc-d311-4781-b2c5-3a14200782cf None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:29.987041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:30.066852 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfcecc-d311-4781-b2c5-3a14200782cf None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:30.107799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f58110-9a7c-4891-b21d-ca3b313e283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:30.108462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f58110-9a7c-4891-b21d-ca3b313e283f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:30.108748 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f58110-9a7c-4891-b21d-ca3b313e283f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:30.109030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f58110-9a7c-4891-b21d-ca3b313e283f None admin] PATH_INFO: `/v3/users/3d90934d388e445184cf04273effcac0` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:30.110202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f58110-9a7c-4891-b21d-ca3b313e283f None admin] RBAC: Authorizing `identity:get_user(user_id=3d90934d388e445184cf04273effcac0)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:30.111784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f58110-9a7c-4891-b21d-ca3b313e283f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:30.113634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 49/102] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:29 2022] GET /identity/v3/users/3d90934d388e445184cf04273effcac0 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:30.119710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f96ed-a661-44f9-bda4-69d36bd1b3c1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:30.201739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:30.258279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f96ed-a661-44f9-bda4-69d36bd1b3c1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:30.308145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca55c2f-6411-4796-ab4f-473f6bb7153f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:30.309145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca55c2f-6411-4796-ab4f-473f6bb7153f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:30.309627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca55c2f-6411-4796-ab4f-473f6bb7153f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:30.310122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca55c2f-6411-4796-ab4f-473f6bb7153f None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:30.311560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca55c2f-6411-4796-ab4f-473f6bb7153f None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:30.313317 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca55c2f-6411-4796-ab4f-473f6bb7153f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:30.316456 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 54/103] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:30 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:30.326205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f58110-9a7c-4891-b21d-ca3b313e283f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:30.452739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:30.468951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f58110-9a7c-4891-b21d-ca3b313e283f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:30.520658 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c9632-6a28-40a1-b12b-fded7ad25804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:30.521358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c9632-6a28-40a1-b12b-fded7ad25804 None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:30.521644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c9632-6a28-40a1-b12b-fded7ad25804 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:30.521921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c9632-6a28-40a1-b12b-fded7ad25804 None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b/users/3d90934d388e445184cf04273effcac0/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:30.524252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c9632-6a28-40a1-b12b-fded7ad25804 None admin] RBAC: Authorizing `identity:create_grant(project_id=290b724bdca948a29018c8ecc35d1c8b, user_id=3d90934d388e445184cf04273effcac0, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:30.526273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c9632-6a28-40a1-b12b-fded7ad25804 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:30.607600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 50/104] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:29:30 2022] PUT /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b/users/3d90934d388e445184cf04273effcac0/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:29:30.636977 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99453617-84d8-4b95-8fd3-457d7da0295d None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:30.637604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99453617-84d8-4b95-8fd3-457d7da0295d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:30.638114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99453617-84d8-4b95-8fd3-457d7da0295d None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:30.640033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 55/105] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:30.654287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219a158-4f14-433a-9ba1-82ecc1026552 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:30.654645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219a158-4f14-433a-9ba1-82ecc1026552 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:30.654928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219a158-4f14-433a-9ba1-82ecc1026552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:30.681520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-b219a158-4f14-433a-9ba1-82ecc1026552 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:30.684066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b219a158-4f14-433a-9ba1-82ecc1026552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:30.775604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1953ed4-ba3d-471d-83d1-89ffa7f6b93b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:30.776255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1953ed4-ba3d-471d-83d1-89ffa7f6b93b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:30.776740 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1953ed4-ba3d-471d-83d1-89ffa7f6b93b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:30.778488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 56/106] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:30.784093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822c2e9-a334-41cf-904e-3148cd758d6c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:30.784676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822c2e9-a334-41cf-904e-3148cd758d6c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:30.785155 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822c2e9-a334-41cf-904e-3148cd758d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:30.790172 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 51/107] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:30.806684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5822c2e9-a334-41cf-904e-3148cd758d6c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:30.809123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5822c2e9-a334-41cf-904e-3148cd758d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:30.905144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 57/108] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:29:31.087550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00e212-28c3-4d3e-a6db-cf6e1cbd073d None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:31.089338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00e212-28c3-4d3e-a6db-cf6e1cbd073d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:31.089845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00e212-28c3-4d3e-a6db-cf6e1cbd073d None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:31.091643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 52/109] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:31.097687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2859b-240f-4586-a0be-57962bf07055 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:31.098285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2859b-240f-4586-a0be-57962bf07055 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:31.098808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2859b-240f-4586-a0be-57962bf07055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:31.119266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e5a2859b-240f-4586-a0be-57962bf07055 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:31.121071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e5a2859b-240f-4586-a0be-57962bf07055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:31.148743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00e212-28c3-4d3e-a6db-cf6e1cbd073d None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:31.218661 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:31.256731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 58/110] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:31.307645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:31.360189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00e212-28c3-4d3e-a6db-cf6e1cbd073d None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:31.412007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e53056-8624-4485-a992-2f4d7acc998a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:31.413122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e53056-8624-4485-a992-2f4d7acc998a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:31.413598 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e53056-8624-4485-a992-2f4d7acc998a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:31.414093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e53056-8624-4485-a992-2f4d7acc998a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:31.427827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e53056-8624-4485-a992-2f4d7acc998a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-45e53056-8624-4485-a992-2f4d7acc998a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:31.434257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:31.441235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:31.442005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 53/111] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:31.444155 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2859b-240f-4586-a0be-57962bf07055 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:31.472996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e53056-8624-4485-a992-2f4d7acc998a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:31.628166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2859b-240f-4586-a0be-57962bf07055 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:31.687919 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e53056-8624-4485-a992-2f4d7acc998a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:31.692547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ecea6-2090-43b3-8d44-c08a2d29129c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:31.693179 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ecea6-2090-43b3-8d44-c08a2d29129c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:31.693466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ecea6-2090-43b3-8d44-c08a2d29129c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:31.693743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ecea6-2090-43b3-8d44-c08a2d29129c None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:31.694443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ecea6-2090-43b3-8d44-c08a2d29129c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:31.694772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ecea6-2090-43b3-8d44-c08a2d29129c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:31.696200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ecea6-2090-43b3-8d44-c08a2d29129c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:31.706544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 59/112] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:31.708266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b184782-f906-44d0-9709-39f3a1e3ee01 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:31.708758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b184782-f906-44d0-9709-39f3a1e3ee01 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:31.709258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b184782-f906-44d0-9709-39f3a1e3ee01 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:31.712405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 60/113] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:31.713777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b184782-f906-44d0-9709-39f3a1e3ee01 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:31.738328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611b412-03c4-49e0-8b57-a9bba4168f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:31.739616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611b412-03c4-49e0-8b57-a9bba4168f54 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:31.740091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611b412-03c4-49e0-8b57-a9bba4168f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:31.740725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611b412-03c4-49e0-8b57-a9bba4168f54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:31.756769 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611b412-03c4-49e0-8b57-a9bba4168f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-f611b412-03c4-49e0-8b57-a9bba4168f54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:31.763829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:31.764750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:31.765519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:31.767826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 54/114] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:29:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:29:31.769000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611b412-03c4-49e0-8b57-a9bba4168f54 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:31.772723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:31.814795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:31.856257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b184782-f906-44d0-9709-39f3a1e3ee01 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:31.892930 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611b412-03c4-49e0-8b57-a9bba4168f54 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:31.893898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faecb54-5e89-4166-ae62-1b0307f21c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:31.894562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecb54-5e89-4166-ae62-1b0307f21c9c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:31.894845 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecb54-5e89-4166-ae62-1b0307f21c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:31.895118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecb54-5e89-4166-ae62-1b0307f21c9c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:31.906684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faecb54-5e89-4166-ae62-1b0307f21c9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-5faecb54-5e89-4166-ae62-1b0307f21c9c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:31.912009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:31.912898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:31.913663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 61/115] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:29:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:31.914762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f17aa-901a-478a-bbd8-13796bae972b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:31.914762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f17aa-901a-478a-bbd8-13796bae972b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:31.914878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f17aa-901a-478a-bbd8-13796bae972b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:31.933895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2b1f17aa-901a-478a-bbd8-13796bae972b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:31.935145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2b1f17aa-901a-478a-bbd8-13796bae972b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:31.940783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155cb4c-035d-4716-8d32-a63c21b34883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:31.941429 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155cb4c-035d-4716-8d32-a63c21b34883 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:31.941707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155cb4c-035d-4716-8d32-a63c21b34883 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:31.941979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155cb4c-035d-4716-8d32-a63c21b34883 None admin] PATH_INFO: `/v3/users/89d0a7f4e32441f9a36c06ff5b3e1ae1` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:31.956786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155cb4c-035d-4716-8d32-a63c21b34883 None admin] RBAC: Authorizing `identity:get_user(user_id=89d0a7f4e32441f9a36c06ff5b3e1ae1)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:31.958285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155cb4c-035d-4716-8d32-a63c21b34883 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:31.960026 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 55/116] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:31 2022] GET /identity/v3/users/89d0a7f4e32441f9a36c06ff5b3e1ae1 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:31.960840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155cb4c-035d-4716-8d32-a63c21b34883 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:32.021004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 62/117] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:32.022805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112ca93-84f8-45b3-9276-096467b1b9e4 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:32.023102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112ca93-84f8-45b3-9276-096467b1b9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:32.023642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112ca93-84f8-45b3-9276-096467b1b9e4 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:32.025179 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 63/118] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:32.025853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112ca93-84f8-45b3-9276-096467b1b9e4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:32.033589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:32.079520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:32.082226 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155cb4c-035d-4716-8d32-a63c21b34883 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:32.121667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdabc52-ec22-46ad-977a-aab00266fd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:32.122453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdabc52-ec22-46ad-977a-aab00266fd73 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:32.122668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdabc52-ec22-46ad-977a-aab00266fd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:32.122944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdabc52-ec22-46ad-977a-aab00266fd73 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:32.123776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdabc52-ec22-46ad-977a-aab00266fd73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:32.124082 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdabc52-ec22-46ad-977a-aab00266fd73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:32.125534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdabc52-ec22-46ad-977a-aab00266fd73 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:32.140728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 56/119] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:32.140728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdabc52-ec22-46ad-977a-aab00266fd73 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:32.160761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112ca93-84f8-45b3-9276-096467b1b9e4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:32.198627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03adf3c-50b3-497f-aeb6-8deb1e20abfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:32.199627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03adf3c-50b3-497f-aeb6-8deb1e20abfe None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:32.200097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03adf3c-50b3-497f-aeb6-8deb1e20abfe None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:32.200570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03adf3c-50b3-497f-aeb6-8deb1e20abfe None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:32.201422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03adf3c-50b3-497f-aeb6-8deb1e20abfe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:32.201915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03adf3c-50b3-497f-aeb6-8deb1e20abfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:32.203543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03adf3c-50b3-497f-aeb6-8deb1e20abfe None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:32.212754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 64/120] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:29:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:32.214862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630578c-ad1c-4538-bcbf-2630f118c893 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:32.215369 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630578c-ad1c-4538-bcbf-2630f118c893 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:32.215838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630578c-ad1c-4538-bcbf-2630f118c893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:32.235662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a630578c-ad1c-4538-bcbf-2630f118c893 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:32.237266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a630578c-ad1c-4538-bcbf-2630f118c893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:32.256909 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdabc52-ec22-46ad-977a-aab00266fd73 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:32.283282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:32.296744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dd9b0-cb83-40ea-b350-b4dc45a46d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:32.297370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dd9b0-cb83-40ea-b350-b4dc45a46d1e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:32.297650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dd9b0-cb83-40ea-b350-b4dc45a46d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:32.302933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dd9b0-cb83-40ea-b350-b4dc45a46d1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:32.317268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 65/121] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:32.318461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630578c-ad1c-4538-bcbf-2630f118c893 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:32.362577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:32.396780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dd9b0-cb83-40ea-b350-b4dc45a46d1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:32.398573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dd9b0-cb83-40ea-b350-b4dc45a46d1e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:32.464759 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630578c-ad1c-4538-bcbf-2630f118c893 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:32.539798 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 57/122] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:32.539798 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dd9b0-cb83-40ea-b350-b4dc45a46d1e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:32.549176 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f87efc-dbea-4276-984d-8222d166bb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:32.550725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f87efc-dbea-4276-984d-8222d166bb40 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:32.551205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f87efc-dbea-4276-984d-8222d166bb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:32.551623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f87efc-dbea-4276-984d-8222d166bb40 None admin] PATH_INFO: `/v3/users/557208629c7f4826b594ef28cabef8ea` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:32.553709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f87efc-dbea-4276-984d-8222d166bb40 None admin] RBAC: Authorizing `identity:get_user(user_id=557208629c7f4826b594ef28cabef8ea)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:32.555697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f87efc-dbea-4276-984d-8222d166bb40 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:32.558371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 66/123] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:32 2022] GET /identity/v3/users/557208629c7f4826b594ef28cabef8ea => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:32.559352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f87efc-dbea-4276-984d-8222d166bb40 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:32.574610 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:32.671250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dd9b0-cb83-40ea-b350-b4dc45a46d1e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:32.674895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:32.679297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f87efc-dbea-4276-984d-8222d166bb40 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:32.706625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0555d92-cde2-41df-999c-19b00ffa14a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:32.707266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0555d92-cde2-41df-999c-19b00ffa14a9 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:32.707575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0555d92-cde2-41df-999c-19b00ffa14a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:32.707851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0555d92-cde2-41df-999c-19b00ffa14a9 None admin] PATH_INFO: `/v3/users/3d90934d388e445184cf04273effcac0` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:32.708990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0555d92-cde2-41df-999c-19b00ffa14a9 None admin] RBAC: Authorizing `identity:get_user(user_id=3d90934d388e445184cf04273effcac0)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:32.710480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0555d92-cde2-41df-999c-19b00ffa14a9 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:32.712083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 58/124] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:32 2022] GET /identity/v3/users/3d90934d388e445184cf04273effcac0 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:32.714673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0555d92-cde2-41df-999c-19b00ffa14a9 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:32.722213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bb964-fd81-4b6c-9f52-b6ff1a0c460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:32.722806 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bb964-fd81-4b6c-9f52-b6ff1a0c460e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:32.723092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bb964-fd81-4b6c-9f52-b6ff1a0c460e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:32.723364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bb964-fd81-4b6c-9f52-b6ff1a0c460e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:32.734816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bb964-fd81-4b6c-9f52-b6ff1a0c460e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a19bb964-fd81-4b6c-9f52-b6ff1a0c460e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:32.739678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:32.740578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:32.741437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 67/125] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:32.742137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bb964-fd81-4b6c-9f52-b6ff1a0c460e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:32.823445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:32.835464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0555d92-cde2-41df-999c-19b00ffa14a9 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:32.851398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bb964-fd81-4b6c-9f52-b6ff1a0c460e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:32.873595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751a655-3a3c-4c6a-8670-dd66752cca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:32.874252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751a655-3a3c-4c6a-8670-dd66752cca1b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:32.874584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751a655-3a3c-4c6a-8670-dd66752cca1b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:32.874864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751a655-3a3c-4c6a-8670-dd66752cca1b None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:32.876006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751a655-3a3c-4c6a-8670-dd66752cca1b None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:32.877484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751a655-3a3c-4c6a-8670-dd66752cca1b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:32.879322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 59/126] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:32 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:32.879935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751a655-3a3c-4c6a-8670-dd66752cca1b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:32.894662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f1a94-53fe-4808-bb62-ca908fea2128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:32.895342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f1a94-53fe-4808-bb62-ca908fea2128 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:32.895630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f1a94-53fe-4808-bb62-ca908fea2128 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:32.895900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f1a94-53fe-4808-bb62-ca908fea2128 None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:32.897028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f1a94-53fe-4808-bb62-ca908fea2128 None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:32.898792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f1a94-53fe-4808-bb62-ca908fea2128 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:32.900744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 68/127] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:32 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:32.901718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f1a94-53fe-4808-bb62-ca908fea2128 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:32.932495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:32.998054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751a655-3a3c-4c6a-8670-dd66752cca1b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:33.009910 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f1a94-53fe-4808-bb62-ca908fea2128 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:33.036636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863d0bd-a8fc-4e7d-ba5b-c89a2896a768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:33.037385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d0bd-a8fc-4e7d-ba5b-c89a2896a768 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:33.037668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d0bd-a8fc-4e7d-ba5b-c89a2896a768 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:33.038003 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d0bd-a8fc-4e7d-ba5b-c89a2896a768 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:33.039221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863d0bd-a8fc-4e7d-ba5b-c89a2896a768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:33.040927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863d0bd-a8fc-4e7d-ba5b-c89a2896a768 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:33.043474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 60/128] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:33.044215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863d0bd-a8fc-4e7d-ba5b-c89a2896a768 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:33.049657 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801f048-3c3c-498a-a798-47988cc0a53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:33.050300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801f048-3c3c-498a-a798-47988cc0a53d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:33.050617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801f048-3c3c-498a-a798-47988cc0a53d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:33.050893 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801f048-3c3c-498a-a798-47988cc0a53d None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:33.052049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801f048-3c3c-498a-a798-47988cc0a53d None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:33.053509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801f048-3c3c-498a-a798-47988cc0a53d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:33.055347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 69/129] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:32 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:33.059371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801f048-3c3c-498a-a798-47988cc0a53d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:33.076840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:33.162310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863d0bd-a8fc-4e7d-ba5b-c89a2896a768 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:33.164050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:33.172319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801f048-3c3c-498a-a798-47988cc0a53d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:33.197888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3a79f-5fe1-4830-97ee-0e713378efc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:33.198653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3a79f-5fe1-4830-97ee-0e713378efc1 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:33.199050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3a79f-5fe1-4830-97ee-0e713378efc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:33.199321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3a79f-5fe1-4830-97ee-0e713378efc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:33.200020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3a79f-5fe1-4830-97ee-0e713378efc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:33.200297 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3a79f-5fe1-4830-97ee-0e713378efc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:33.201668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3a79f-5fe1-4830-97ee-0e713378efc1 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:33.207249 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c2d29-aabd-4cbc-bd84-cf4c3951a966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:33.207859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c2d29-aabd-4cbc-bd84-cf4c3951a966 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:33.208141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c2d29-aabd-4cbc-bd84-cf4c3951a966 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:33.208412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c2d29-aabd-4cbc-bd84-cf4c3951a966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:33.209139 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c2d29-aabd-4cbc-bd84-cf4c3951a966 None admin] RBAC: Adding query filter params (role.id=6f434b187a404b7cb1e9bb8303ce59b5, scope.project.id=290b724bdca948a29018c8ecc35d1c8b, user.id=89d0a7f4e32441f9a36c06ff5b3e1ae1) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:33.209451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c2d29-aabd-4cbc-bd84-cf4c3951a966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:33.210074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 61/130] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:33.210903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3a79f-5fe1-4830-97ee-0e713378efc1 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:33.210994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c2d29-aabd-4cbc-bd84-cf4c3951a966 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:33.221090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 70/131] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:33 2022] GET /identity/v3/role_assignments?user.id=89d0a7f4e32441f9a36c06ff5b3e1ae1&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=6f434b187a404b7cb1e9bb8303ce59b5 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:33.222467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c2d29-aabd-4cbc-bd84-cf4c3951a966 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:33.333654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:33.354136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c2d29-aabd-4cbc-bd84-cf4c3951a966 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:33.354281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3a79f-5fe1-4830-97ee-0e713378efc1 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:33.393473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487fceb-f14a-42fe-8741-f30508ab0309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:33.393692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32034f-0352-46a8-b35f-081c1d29533c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:33.394093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487fceb-f14a-42fe-8741-f30508ab0309 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:33.394298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32034f-0352-46a8-b35f-081c1d29533c None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:33.394391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487fceb-f14a-42fe-8741-f30508ab0309 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:33.394597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32034f-0352-46a8-b35f-081c1d29533c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:33.394676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487fceb-f14a-42fe-8741-f30508ab0309 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:33.394884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32034f-0352-46a8-b35f-081c1d29533c None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b/users/557208629c7f4826b594ef28cabef8ea/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:33.395314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487fceb-f14a-42fe-8741-f30508ab0309 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:33.396792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487fceb-f14a-42fe-8741-f30508ab0309 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:33.396898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32034f-0352-46a8-b35f-081c1d29533c None admin] RBAC: Authorizing `identity:create_grant(project_id=290b724bdca948a29018c8ecc35d1c8b, user_id=557208629c7f4826b594ef28cabef8ea, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:33.398644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32034f-0352-46a8-b35f-081c1d29533c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:33.513838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 62/132] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:29:33 2022] PUT /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b/users/557208629c7f4826b594ef28cabef8ea/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:29:33.514705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32034f-0352-46a8-b35f-081c1d29533c None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:33.538781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 71/133] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:29:33 2022] POST /identity/v3/projects => generated 302 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:33.540589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46192a08-d17a-4f4d-aeee-d22c752ba6cb None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:33.542022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46192a08-d17a-4f4d-aeee-d22c752ba6cb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:33.542353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46192a08-d17a-4f4d-aeee-d22c752ba6cb None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:33.543913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 72/134] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:33.549025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46192a08-d17a-4f4d-aeee-d22c752ba6cb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:33.578889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:33.709365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32034f-0352-46a8-b35f-081c1d29533c None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:33.756601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02777c8-4f7a-4b56-af15-7bbdbc9b835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:33.757616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02777c8-4f7a-4b56-af15-7bbdbc9b835c None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:33.758167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02777c8-4f7a-4b56-af15-7bbdbc9b835c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:33.758535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02777c8-4f7a-4b56-af15-7bbdbc9b835c None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b/users/3d90934d388e445184cf04273effcac0/roles/a38face2eeae40c28edda83a4d46614f` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:33.762463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02777c8-4f7a-4b56-af15-7bbdbc9b835c None admin] RBAC: Authorizing `identity:create_grant(project_id=290b724bdca948a29018c8ecc35d1c8b, user_id=3d90934d388e445184cf04273effcac0, role_id=a38face2eeae40c28edda83a4d46614f)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:33.764328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02777c8-4f7a-4b56-af15-7bbdbc9b835c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:33.767975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46192a08-d17a-4f4d-aeee-d22c752ba6cb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:33.812080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3173e-7709-4d63-af73-a6d5a060acae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:33.812723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3173e-7709-4d63-af73-a6d5a060acae None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:33.813004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3173e-7709-4d63-af73-a6d5a060acae None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:33.813284 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3173e-7709-4d63-af73-a6d5a060acae None admin] PATH_INFO: `/v3/users/3d90934d388e445184cf04273effcac0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:33.814462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3173e-7709-4d63-af73-a6d5a060acae None admin] RBAC: Authorizing `identity:get_user(user_id=3d90934d388e445184cf04273effcac0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:33.815944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3173e-7709-4d63-af73-a6d5a060acae None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:33.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 73/135] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:33 2022] GET /identity/v3/users/3d90934d388e445184cf04273effcac0 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:33.819649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce941f-74b5-4a3c-9ba1-2126d656aa77 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:33.820130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce941f-74b5-4a3c-9ba1-2126d656aa77 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:33.820617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce941f-74b5-4a3c-9ba1-2126d656aa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:33.843755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1cce941f-74b5-4a3c-9ba1-2126d656aa77 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:33.845285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1cce941f-74b5-4a3c-9ba1-2126d656aa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:33.859415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 63/136] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:29:33 2022] PUT /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b/users/3d90934d388e445184cf04273effcac0/roles/a38face2eeae40c28edda83a4d46614f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:29:33.860721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02777c8-4f7a-4b56-af15-7bbdbc9b835c None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:33.885945 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:33.983345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 74/137] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:34.058477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02777c8-4f7a-4b56-af15-7bbdbc9b835c None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:34.101893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49ed4e-aea8-426d-827b-47e4fb19fcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:34.102573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49ed4e-aea8-426d-827b-47e4fb19fcde None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:34.102863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49ed4e-aea8-426d-827b-47e4fb19fcde None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:34.103146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49ed4e-aea8-426d-827b-47e4fb19fcde None admin] PATH_INFO: `/v3/projects/955d8cd8cc0f40768c3327e1ced5b79b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:34.104869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49ed4e-aea8-426d-827b-47e4fb19fcde None admin] RBAC: Authorizing `identity:get_project(project_id=955d8cd8cc0f40768c3327e1ced5b79b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:34.106340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49ed4e-aea8-426d-827b-47e4fb19fcde None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:34.108183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 64/138] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:33 2022] GET /identity/v3/projects/955d8cd8cc0f40768c3327e1ced5b79b => generated 312 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:34.114274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce941f-74b5-4a3c-9ba1-2126d656aa77 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:34.136145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:34.253813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce941f-74b5-4a3c-9ba1-2126d656aa77 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:34.303302 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0b3d0-fe22-4a8f-9143-b951d9e4ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:34.303985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0b3d0-fe22-4a8f-9143-b951d9e4ff12 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:34.304266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0b3d0-fe22-4a8f-9143-b951d9e4ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:34.304540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0b3d0-fe22-4a8f-9143-b951d9e4ff12 None admin] PATH_INFO: `/v3/projects/955d8cd8cc0f40768c3327e1ced5b79b/users/3d90934d388e445184cf04273effcac0/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:34.307108 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0b3d0-fe22-4a8f-9143-b951d9e4ff12 None admin] RBAC: Authorizing `identity:create_grant(project_id=955d8cd8cc0f40768c3327e1ced5b79b, user_id=3d90934d388e445184cf04273effcac0, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:34.308770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0b3d0-fe22-4a8f-9143-b951d9e4ff12 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:34.383354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 75/139] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:29:34 2022] PUT /identity/v3/projects/955d8cd8cc0f40768c3327e1ced5b79b/users/3d90934d388e445184cf04273effcac0/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:29:34.519377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49ed4e-aea8-426d-827b-47e4fb19fcde None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:34.604120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:34.637461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49ed4e-aea8-426d-827b-47e4fb19fcde None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:34.682191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43467fe0-ee53-499e-b549-8429864bee64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:34.682877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43467fe0-ee53-499e-b549-8429864bee64 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:34.683172 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43467fe0-ee53-499e-b549-8429864bee64 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:34.683431 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43467fe0-ee53-499e-b549-8429864bee64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:34.701794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43467fe0-ee53-499e-b549-8429864bee64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-43467fe0-ee53-499e-b549-8429864bee64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:34.707832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:34.708735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:34.709584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:34.712064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 65/140] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:34.718043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0b3d0-fe22-4a8f-9143-b951d9e4ff12 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:34.851894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:34.896594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0b3d0-fe22-4a8f-9143-b951d9e4ff12 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:34.945098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295827-60f6-44b0-a0d2-c96ef8c618b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:34.945724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295827-60f6-44b0-a0d2-c96ef8c618b1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:34.946000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295827-60f6-44b0-a0d2-c96ef8c618b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:34.946267 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295827-60f6-44b0-a0d2-c96ef8c618b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:34.947581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab295827-60f6-44b0-a0d2-c96ef8c618b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:34.947927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab295827-60f6-44b0-a0d2-c96ef8c618b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:34.949714 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab295827-60f6-44b0-a0d2-c96ef8c618b1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:34.962522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 76/141] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:34.968937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43467fe0-ee53-499e-b549-8429864bee64 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:35.103310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43467fe0-ee53-499e-b549-8429864bee64 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:35.151108 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd24ae-70f1-4053-b6db-c2c6c9fbb2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:35.152119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd24ae-70f1-4053-b6db-c2c6c9fbb2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:35.152600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd24ae-70f1-4053-b6db-c2c6c9fbb2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:35.153143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd24ae-70f1-4053-b6db-c2c6c9fbb2d1 None admin] PATH_INFO: `/v3/users/3d90934d388e445184cf04273effcac0` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:35.159167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd24ae-70f1-4053-b6db-c2c6c9fbb2d1 None admin] RBAC: Authorizing `identity:get_user(user_id=3d90934d388e445184cf04273effcac0)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:35.160917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd24ae-70f1-4053-b6db-c2c6c9fbb2d1 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:35.162800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 66/142] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:34 2022] GET /identity/v3/users/3d90934d388e445184cf04273effcac0 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:35.169238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295827-60f6-44b0-a0d2-c96ef8c618b1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:35.365445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295827-60f6-44b0-a0d2-c96ef8c618b1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:35.417288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:35.424708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8272e12-b4ed-431e-94f4-30e978b4dc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:35.425344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8272e12-b4ed-431e-94f4-30e978b4dc90 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:35.425619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8272e12-b4ed-431e-94f4-30e978b4dc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:35.425888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8272e12-b4ed-431e-94f4-30e978b4dc90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:35.519275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8272e12-b4ed-431e-94f4-30e978b4dc90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:35.521122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8272e12-b4ed-431e-94f4-30e978b4dc90 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:35.631832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 77/143] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:35.638153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd24ae-70f1-4053-b6db-c2c6c9fbb2d1 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:35.647624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:35.765446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa00d0-37db-4412-9a93-16b0d8952a09 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:35.765835 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa00d0-37db-4412-9a93-16b0d8952a09 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:35.766010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa00d0-37db-4412-9a93-16b0d8952a09 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:35.767783 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 78/144] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:29:35.778316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0351b63-ecd7-462e-a752-eb6acb13f26a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:35.778641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0351b63-ecd7-462e-a752-eb6acb13f26a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:35.778920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0351b63-ecd7-462e-a752-eb6acb13f26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:35.783858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd24ae-70f1-4053-b6db-c2c6c9fbb2d1 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:35.810387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b0351b63-ecd7-462e-a752-eb6acb13f26a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:35.811736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b0351b63-ecd7-462e-a752-eb6acb13f26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:35.822655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36adc8-9044-4dd9-9804-3505f11c9be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:35.823311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36adc8-9044-4dd9-9804-3505f11c9be0 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:35.823593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36adc8-9044-4dd9-9804-3505f11c9be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:35.823872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36adc8-9044-4dd9-9804-3505f11c9be0 None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:35.825048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36adc8-9044-4dd9-9804-3505f11c9be0 None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:35.826531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36adc8-9044-4dd9-9804-3505f11c9be0 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:35.828183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 67/145] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:35 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:35.834445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36adc8-9044-4dd9-9804-3505f11c9be0 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:35.868781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:35.967878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 79/146] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:35.995588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:36.051012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36adc8-9044-4dd9-9804-3505f11c9be0 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:36.112781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c650b-5486-4979-9c01-5f0b4befcbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:36.113514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c650b-5486-4979-9c01-5f0b4befcbd3 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:36.113806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c650b-5486-4979-9c01-5f0b4befcbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:36.114085 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c650b-5486-4979-9c01-5f0b4befcbd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:36.114866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c650b-5486-4979-9c01-5f0b4befcbd3 None admin] RBAC: Adding query filter params (role.id=4d98a2b16f8f42bca3e5dac0374cdb54, scope.project.id=290b724bdca948a29018c8ecc35d1c8b, user.id=3d90934d388e445184cf04273effcac0) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:36.115183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c650b-5486-4979-9c01-5f0b4befcbd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:36.116650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c650b-5486-4979-9c01-5f0b4befcbd3 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:36.125585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 68/147] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:35 2022] GET /identity/v3/role_assignments?user.id=3d90934d388e445184cf04273effcac0&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:36.441208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0351b63-ecd7-462e-a752-eb6acb13f26a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:36.589888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0351b63-ecd7-462e-a752-eb6acb13f26a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:36.638151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8b1b3-53b4-48b1-afea-3236aecb6d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:36.638804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8b1b3-53b4-48b1-afea-3236aecb6d6d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:36.639085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8b1b3-53b4-48b1-afea-3236aecb6d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:36.639360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8b1b3-53b4-48b1-afea-3236aecb6d6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:36.643899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abe5d6-3639-4b47-9f9e-e4dd91a77741 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:36.644550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abe5d6-3639-4b47-9f9e-e4dd91a77741 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:36.644707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abe5d6-3639-4b47-9f9e-e4dd91a77741 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:36.646007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 69/148] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:29:36.653561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9cb4d-0613-4fe7-9cbb-efd6eb698c02 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:36.654194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9cb4d-0613-4fe7-9cbb-efd6eb698c02 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:36.654633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9cb4d-0613-4fe7-9cbb-efd6eb698c02 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:36.656174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 70/149] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:29:36.658141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8b1b3-53b4-48b1-afea-3236aecb6d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:36.658511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac292b5b-5f98-400a-83f2-1a7c084e4eaa None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:36.659048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac292b5b-5f98-400a-83f2-1a7c084e4eaa None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:36.659536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac292b5b-5f98-400a-83f2-1a7c084e4eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-b6f8b1b3-53b4-48b1-afea-3236aecb6d6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:36.663183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:36.664149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 80/150] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:36.666955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc53d9-011d-4fef-9bd2-2119491163f3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:36.667705 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc53d9-011d-4fef-9bd2-2119491163f3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:36.667705 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc53d9-011d-4fef-9bd2-2119491163f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:36.686902 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9bfc53d9-011d-4fef-9bd2-2119491163f3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:36.687115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-ac292b5b-5f98-400a-83f2-1a7c084e4eaa None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:36.688142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9bfc53d9-011d-4fef-9bd2-2119491163f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:36.688256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ac292b5b-5f98-400a-83f2-1a7c084e4eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:36.704710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:36.765490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 81/151] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:36.765918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 71/152] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:36.766365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc53d9-011d-4fef-9bd2-2119491163f3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:36.855181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba56db-7cc0-46c9-a818-cd6b8bd4ce13 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:36.855467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba56db-7cc0-46c9-a818-cd6b8bd4ce13 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:36.855744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba56db-7cc0-46c9-a818-cd6b8bd4ce13 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:36.857371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 72/153] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:36.872051 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176e4e0-7fed-405b-83ed-c8a274e8bbe5 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:36.872338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176e4e0-7fed-405b-83ed-c8a274e8bbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:36.872631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176e4e0-7fed-405b-83ed-c8a274e8bbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:36.908858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-1176e4e0-7fed-405b-83ed-c8a274e8bbe5 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:36.908858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1176e4e0-7fed-405b-83ed-c8a274e8bbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:36.917807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc53d9-011d-4fef-9bd2-2119491163f3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:36.960904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f459f6-80b3-47e7-b6ad-b5c890b7bd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:36.962130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f459f6-80b3-47e7-b6ad-b5c890b7bd0b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:36.963184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f459f6-80b3-47e7-b6ad-b5c890b7bd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:36.963649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f459f6-80b3-47e7-b6ad-b5c890b7bd0b None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:36.964526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f459f6-80b3-47e7-b6ad-b5c890b7bd0b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:36.965018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f459f6-80b3-47e7-b6ad-b5c890b7bd0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:36.966757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f459f6-80b3-47e7-b6ad-b5c890b7bd0b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:36.977834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 82/154] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:36.987694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f459f6-80b3-47e7-b6ad-b5c890b7bd0b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:37.006759 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:37.040895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:37.056986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 73/155] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:29:37.096599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176e4e0-7fed-405b-83ed-c8a274e8bbe5 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:37.113381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f459f6-80b3-47e7-b6ad-b5c890b7bd0b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:37.171807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a9945-0dbf-4860-8081-8ab4d163596a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:37.172370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a9945-0dbf-4860-8081-8ab4d163596a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:37.172649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a9945-0dbf-4860-8081-8ab4d163596a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:37.172923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a9945-0dbf-4860-8081-8ab4d163596a None admin] PATH_INFO: `/v3/users/89d0a7f4e32441f9a36c06ff5b3e1ae1` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:37.179134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a9945-0dbf-4860-8081-8ab4d163596a None admin] RBAC: Authorizing `identity:get_user(user_id=89d0a7f4e32441f9a36c06ff5b3e1ae1)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:37.180688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a9945-0dbf-4860-8081-8ab4d163596a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:37.182624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 83/156] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:36 2022] GET /identity/v3/users/89d0a7f4e32441f9a36c06ff5b3e1ae1 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:37.192394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a9945-0dbf-4860-8081-8ab4d163596a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:37.211725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176e4e0-7fed-405b-83ed-c8a274e8bbe5 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:37.250366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9486ba2-42b0-49d8-b553-86fbd42f4252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:37.251034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9486ba2-42b0-49d8-b553-86fbd42f4252 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:37.251314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9486ba2-42b0-49d8-b553-86fbd42f4252 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:37.251587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9486ba2-42b0-49d8-b553-86fbd42f4252 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:37.252782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9486ba2-42b0-49d8-b553-86fbd42f4252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:37.254353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9486ba2-42b0-49d8-b553-86fbd42f4252 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:37.256016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 74/157] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:37.256802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9486ba2-42b0-49d8-b553-86fbd42f4252 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:37.307125 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:37.313308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a9945-0dbf-4860-8081-8ab4d163596a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:37.325404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:37.362688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3689429-2e4e-427c-909b-4324d4ff6f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:37.363635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3689429-2e4e-427c-909b-4324d4ff6f0a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:37.364057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3689429-2e4e-427c-909b-4324d4ff6f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:37.364525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3689429-2e4e-427c-909b-4324d4ff6f0a None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:37.365807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3689429-2e4e-427c-909b-4324d4ff6f0a None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:37.367471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3689429-2e4e-427c-909b-4324d4ff6f0a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:37.373355 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9486ba2-42b0-49d8-b553-86fbd42f4252 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:37.373499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 84/158] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:37 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:37.373499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3689429-2e4e-427c-909b-4324d4ff6f0a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:37.408599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e58e8f-03be-4320-b8b5-c134010d670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:37.409383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e58e8f-03be-4320-b8b5-c134010d670b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:37.409553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e58e8f-03be-4320-b8b5-c134010d670b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:37.409817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e58e8f-03be-4320-b8b5-c134010d670b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:37.422499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e58e8f-03be-4320-b8b5-c134010d670b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-b0e58e8f-03be-4320-b8b5-c134010d670b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:37.427388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:37.428424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:37.429178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 75/159] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:29:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:37.429932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e58e8f-03be-4320-b8b5-c134010d670b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:37.495553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3689429-2e4e-427c-909b-4324d4ff6f0a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:37.543276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e58e8f-03be-4320-b8b5-c134010d670b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:37.545873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83ff3a-bc4e-45e7-b775-9c86164afe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:37.546522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83ff3a-bc4e-45e7-b775-9c86164afe80 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:37.546800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83ff3a-bc4e-45e7-b775-9c86164afe80 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:37.547074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83ff3a-bc4e-45e7-b775-9c86164afe80 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:37.547726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83ff3a-bc4e-45e7-b775-9c86164afe80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:37.549185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83ff3a-bc4e-45e7-b775-9c86164afe80 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:37.567253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:37.578477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80b604-88ca-4c1c-bc68-bf56bc1b000a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:37.579146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80b604-88ca-4c1c-bc68-bf56bc1b000a None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:37.579425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80b604-88ca-4c1c-bc68-bf56bc1b000a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:37.579699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80b604-88ca-4c1c-bc68-bf56bc1b000a None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b/users/89d0a7f4e32441f9a36c06ff5b3e1ae1/roles/6f434b187a404b7cb1e9bb8303ce59b5` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:37.581734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80b604-88ca-4c1c-bc68-bf56bc1b000a None admin] RBAC: Authorizing `identity:create_grant(project_id=290b724bdca948a29018c8ecc35d1c8b, user_id=89d0a7f4e32441f9a36c06ff5b3e1ae1, role_id=6f434b187a404b7cb1e9bb8303ce59b5)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:37.583224 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80b604-88ca-4c1c-bc68-bf56bc1b000a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:37.656470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 85/160] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:29:37 2022] POST /identity/v3/users => generated 283 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:37.657387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83ff3a-bc4e-45e7-b775-9c86164afe80 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:37.660915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:37.686574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 76/161] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:29:37 2022] PUT /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b/users/89d0a7f4e32441f9a36c06ff5b3e1ae1/roles/6f434b187a404b7cb1e9bb8303ce59b5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:29:37.686574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80b604-88ca-4c1c-bc68-bf56bc1b000a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:37.814948 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83ff3a-bc4e-45e7-b775-9c86164afe80 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:37.817315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80b604-88ca-4c1c-bc68-bf56bc1b000a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:37.868410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893fac5-4cd0-473f-a5b2-00c022bcda21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:37.869336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893fac5-4cd0-473f-a5b2-00c022bcda21 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:37.869673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893fac5-4cd0-473f-a5b2-00c022bcda21 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:37.869934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893fac5-4cd0-473f-a5b2-00c022bcda21 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:37.872908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfbb0d-8838-46c1-b69e-380ce2853089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:37.873529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfbb0d-8838-46c1-b69e-380ce2853089 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:37.873811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfbb0d-8838-46c1-b69e-380ce2853089 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:37.874086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfbb0d-8838-46c1-b69e-380ce2853089 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:37.874798 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfbb0d-8838-46c1-b69e-380ce2853089 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:37.875104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfbb0d-8838-46c1-b69e-380ce2853089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:37.876499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfbb0d-8838-46c1-b69e-380ce2853089 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:37.889603 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 77/162] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:37.891346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9f98-f8e2-402d-a9b5-28bca257ea06 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:37.891854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9f98-f8e2-402d-a9b5-28bca257ea06 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:37.892331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9f98-f8e2-402d-a9b5-28bca257ea06 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:37.893839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 78/163] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:37.894786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893fac5-4cd0-473f-a5b2-00c022bcda21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:37.895045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c9f98-f8e2-402d-a9b5-28bca257ea06 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1893fac5-4cd0-473f-a5b2-00c022bcda21 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:37.901062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:37.902021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:37.902924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 86/164] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:29:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:37.903704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd621ae-9c95-4ab8-9d8a-02707f30a050 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:37.903785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd621ae-9c95-4ab8-9d8a-02707f30a050 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:37.904662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd621ae-9c95-4ab8-9d8a-02707f30a050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:37.930164 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-bdd621ae-9c95-4ab8-9d8a-02707f30a050 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:37.933177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bdd621ae-9c95-4ab8-9d8a-02707f30a050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:37.965333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:38.037744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:38.045284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c9f98-f8e2-402d-a9b5-28bca257ea06 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:38.048138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 87/165] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:38.048850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd621ae-9c95-4ab8-9d8a-02707f30a050 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:38.088600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69701a80-ab72-4745-8bc3-c01af98a05cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:38.089254 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69701a80-ab72-4745-8bc3-c01af98a05cc None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:38.089533 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69701a80-ab72-4745-8bc3-c01af98a05cc None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:38.089868 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69701a80-ab72-4745-8bc3-c01af98a05cc None admin] PATH_INFO: `/v3/users/557208629c7f4826b594ef28cabef8ea` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:38.091573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69701a80-ab72-4745-8bc3-c01af98a05cc None admin] RBAC: Authorizing `identity:get_user(user_id=557208629c7f4826b594ef28cabef8ea)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:38.093036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69701a80-ab72-4745-8bc3-c01af98a05cc None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:38.094664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 79/166] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:37 2022] GET /identity/v3/users/557208629c7f4826b594ef28cabef8ea => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:38.100379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69701a80-ab72-4745-8bc3-c01af98a05cc None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:38.191528 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd621ae-9c95-4ab8-9d8a-02707f30a050 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:38.215758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69701a80-ab72-4745-8bc3-c01af98a05cc None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:38.231861 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:38.254302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1ee7d-747d-4b0e-ac9e-1124bdf70d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:38.255129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1037ff-87eb-49c2-ac95-1c6ffc07666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:38.255651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1ee7d-747d-4b0e-ac9e-1124bdf70d60 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:38.255765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1037ff-87eb-49c2-ac95-1c6ffc07666c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:38.256045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1037ff-87eb-49c2-ac95-1c6ffc07666c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:38.256309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1037ff-87eb-49c2-ac95-1c6ffc07666c None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:38.256663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1ee7d-747d-4b0e-ac9e-1124bdf70d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:38.256984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1037ff-87eb-49c2-ac95-1c6ffc07666c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:38.257284 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1037ff-87eb-49c2-ac95-1c6ffc07666c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:38.257525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1ee7d-747d-4b0e-ac9e-1124bdf70d60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:38.259420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1037ff-87eb-49c2-ac95-1c6ffc07666c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:38.268433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 88/167] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:29:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:38.274307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1037ff-87eb-49c2-ac95-1c6ffc07666c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:38.337470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1ee7d-747d-4b0e-ac9e-1124bdf70d60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:38.339511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1ee7d-747d-4b0e-ac9e-1124bdf70d60 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:38.375501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:38.453734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 80/168] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 23:29:38.460109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1ee7d-747d-4b0e-ac9e-1124bdf70d60 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:38.473730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1037ff-87eb-49c2-ac95-1c6ffc07666c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:38.489503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:38.542686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78534ecb-ed2d-4bbb-bc7d-f9c5885d0177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:38.543378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534ecb-ed2d-4bbb-bc7d-f9c5885d0177 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:38.543658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534ecb-ed2d-4bbb-bc7d-f9c5885d0177 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:38.543929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534ecb-ed2d-4bbb-bc7d-f9c5885d0177 None admin] PATH_INFO: `/v3/users/3d90934d388e445184cf04273effcac0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:38.546365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78534ecb-ed2d-4bbb-bc7d-f9c5885d0177 None admin] RBAC: Authorizing `identity:get_user(user_id=3d90934d388e445184cf04273effcac0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:38.548514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78534ecb-ed2d-4bbb-bc7d-f9c5885d0177 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:38.550340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 89/169] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:38 2022] GET /identity/v3/users/3d90934d388e445184cf04273effcac0 => generated 275 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:38.551247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78534ecb-ed2d-4bbb-bc7d-f9c5885d0177 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:38.611827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1ee7d-747d-4b0e-ac9e-1124bdf70d60 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:38.653410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d081170-2e29-473d-985b-bdaef916e0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:38.654050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d081170-2e29-473d-985b-bdaef916e0fd None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:38.654327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d081170-2e29-473d-985b-bdaef916e0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:38.655280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d081170-2e29-473d-985b-bdaef916e0fd None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:38.656589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d081170-2e29-473d-985b-bdaef916e0fd None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:38.658054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d081170-2e29-473d-985b-bdaef916e0fd None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:38.662612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 81/170] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:38 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:38.663528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d081170-2e29-473d-985b-bdaef916e0fd None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:38.697887 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78534ecb-ed2d-4bbb-bc7d-f9c5885d0177 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:38.728730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:38.740221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a666a8-6b31-41a9-ac3f-bc2dc9220323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:38.742223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a666a8-6b31-41a9-ac3f-bc2dc9220323 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:38.742223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a666a8-6b31-41a9-ac3f-bc2dc9220323 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:38.742223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a666a8-6b31-41a9-ac3f-bc2dc9220323 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:38.755226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a666a8-6b31-41a9-ac3f-bc2dc9220323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-f6a666a8-6b31-41a9-ac3f-bc2dc9220323 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:38.761105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:38.762016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:38.762894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 90/171] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:38.763558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a666a8-6b31-41a9-ac3f-bc2dc9220323 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:38.856832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:38.893068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d081170-2e29-473d-985b-bdaef916e0fd None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:38.988185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285ae2f-1da3-4c7c-bb02-1ad884ee22d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:38.988714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285ae2f-1da3-4c7c-bb02-1ad884ee22d1 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:38.988990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285ae2f-1da3-4c7c-bb02-1ad884ee22d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:38.989255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285ae2f-1da3-4c7c-bb02-1ad884ee22d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:39.004577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a666a8-6b31-41a9-ac3f-bc2dc9220323 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:39.100166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b1931-9d83-48a5-90be-152f0c709658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:39.100794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b1931-9d83-48a5-90be-152f0c709658 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:39.101076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b1931-9d83-48a5-90be-152f0c709658 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:39.101348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b1931-9d83-48a5-90be-152f0c709658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:39.102088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b1931-9d83-48a5-90be-152f0c709658 None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=290b724bdca948a29018c8ecc35d1c8b, user.id=557208629c7f4826b594ef28cabef8ea) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:39.102401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b1931-9d83-48a5-90be-152f0c709658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:39.104728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b1931-9d83-48a5-90be-152f0c709658 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:39.111576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 91/172] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:38 2022] GET /identity/v3/role_assignments?user.id=557208629c7f4826b594ef28cabef8ea&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=d86dba8efe20423abdde06031690beb8 => generated 608 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:39.112801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b1931-9d83-48a5-90be-152f0c709658 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:39.195691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285ae2f-1da3-4c7c-bb02-1ad884ee22d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:39.198008 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285ae2f-1da3-4c7c-bb02-1ad884ee22d1 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:39.210810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:39.300134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b1931-9d83-48a5-90be-152f0c709658 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:39.352924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37947f9d-4271-4e27-a3d5-07dc6b1d0dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:39.353557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37947f9d-4271-4e27-a3d5-07dc6b1d0dee None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:39.353838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37947f9d-4271-4e27-a3d5-07dc6b1d0dee None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:39.354112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37947f9d-4271-4e27-a3d5-07dc6b1d0dee None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:39.354817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37947f9d-4271-4e27-a3d5-07dc6b1d0dee None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:39.355125 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37947f9d-4271-4e27-a3d5-07dc6b1d0dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:39.357047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37947f9d-4271-4e27-a3d5-07dc6b1d0dee None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:39.374612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 92/173] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:39.386239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37947f9d-4271-4e27-a3d5-07dc6b1d0dee None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:39.445442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:39.481059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 82/174] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 818 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:39.487105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285ae2f-1da3-4c7c-bb02-1ad884ee22d1 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:39.563390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37947f9d-4271-4e27-a3d5-07dc6b1d0dee None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:39.578032 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:39.626541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75999eeb-ba38-4343-8edb-d08b58bfb4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:39.627184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75999eeb-ba38-4343-8edb-d08b58bfb4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:39.627469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75999eeb-ba38-4343-8edb-d08b58bfb4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:39.627744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75999eeb-ba38-4343-8edb-d08b58bfb4e7 None admin] PATH_INFO: `/v3/users/3d90934d388e445184cf04273effcac0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:39.635342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75999eeb-ba38-4343-8edb-d08b58bfb4e7 None admin] RBAC: Authorizing `identity:get_user(user_id=3d90934d388e445184cf04273effcac0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:39.636820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75999eeb-ba38-4343-8edb-d08b58bfb4e7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:39.638669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 93/175] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:39 2022] GET /identity/v3/users/3d90934d388e445184cf04273effcac0 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:39.644325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75999eeb-ba38-4343-8edb-d08b58bfb4e7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:39.657204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285ae2f-1da3-4c7c-bb02-1ad884ee22d1 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:39.701951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700dc9b-8cd3-4164-b056-c03781310e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:39.702616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700dc9b-8cd3-4164-b056-c03781310e50 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:39.702897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700dc9b-8cd3-4164-b056-c03781310e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:39.703167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700dc9b-8cd3-4164-b056-c03781310e50 None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:39.704443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700dc9b-8cd3-4164-b056-c03781310e50 None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:39.705893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700dc9b-8cd3-4164-b056-c03781310e50 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:39.709778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 83/176] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:39 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:39.717364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700dc9b-8cd3-4164-b056-c03781310e50 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:39.782638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:39.819561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75999eeb-ba38-4343-8edb-d08b58bfb4e7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:39.836296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700dc9b-8cd3-4164-b056-c03781310e50 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:39.873062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4800a-5f1d-430b-a55d-52d3b5bde430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:39.873691 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a18318-68dc-47f6-a629-e445e9f5f7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:39.873782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4800a-5f1d-430b-a55d-52d3b5bde430 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:39.874088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4800a-5f1d-430b-a55d-52d3b5bde430 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:39.874185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a18318-68dc-47f6-a629-e445e9f5f7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:39.874254 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4800a-5f1d-430b-a55d-52d3b5bde430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:39.874448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a18318-68dc-47f6-a629-e445e9f5f7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:39.874979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4800a-5f1d-430b-a55d-52d3b5bde430 None admin] RBAC: Adding query filter params (role.id=a38face2eeae40c28edda83a4d46614f, scope.project.id=290b724bdca948a29018c8ecc35d1c8b, user.id=3d90934d388e445184cf04273effcac0) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:39.875287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4800a-5f1d-430b-a55d-52d3b5bde430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:39.876727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4800a-5f1d-430b-a55d-52d3b5bde430 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:39.882830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a18318-68dc-47f6-a629-e445e9f5f7d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:39.882971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 84/177] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:39 2022] GET /identity/v3/role_assignments?user.id=3d90934d388e445184cf04273effcac0&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=a38face2eeae40c28edda83a4d46614f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:39.962940 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:39.987209 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a18318-68dc-47f6-a629-e445e9f5f7d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:39.988765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a18318-68dc-47f6-a629-e445e9f5f7d1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:40.096986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 94/178] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 23:29:40.106059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4800a-5f1d-430b-a55d-52d3b5bde430 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:40.162105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c038c-2d94-4a08-a9d9-deba5240ee69 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:40.162724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c038c-2d94-4a08-a9d9-deba5240ee69 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:40.163230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c038c-2d94-4a08-a9d9-deba5240ee69 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:40.165066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 95/179] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:40.171999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a8213-1cc5-406b-b73d-e7c18a9efcc5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:40.172291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a8213-1cc5-406b-b73d-e7c18a9efcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:40.172558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a8213-1cc5-406b-b73d-e7c18a9efcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:40.191623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f04a8213-1cc5-406b-b73d-e7c18a9efcc5 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:40.192913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f04a8213-1cc5-406b-b73d-e7c18a9efcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:40.267114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 96/180] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:29:40.279238 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4800a-5f1d-430b-a55d-52d3b5bde430 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:40.314061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:40.325411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2e2ef-c29f-49ac-8476-b37af952d398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:40.325982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2e2ef-c29f-49ac-8476-b37af952d398 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:40.326265 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2e2ef-c29f-49ac-8476-b37af952d398 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:40.326576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2e2ef-c29f-49ac-8476-b37af952d398 None admin] PATH_INFO: `/v3/projects/955d8cd8cc0f40768c3327e1ced5b79b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:40.327782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2e2ef-c29f-49ac-8476-b37af952d398 None admin] RBAC: Authorizing `identity:get_project(project_id=955d8cd8cc0f40768c3327e1ced5b79b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:40.329234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2e2ef-c29f-49ac-8476-b37af952d398 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:40.330906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 85/181] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:40 2022] GET /identity/v3/projects/955d8cd8cc0f40768c3327e1ced5b79b => generated 312 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:40.337081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a8213-1cc5-406b-b73d-e7c18a9efcc5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:40.350802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:40.474730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a8213-1cc5-406b-b73d-e7c18a9efcc5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:40.518648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c02aa1-802d-479d-ae6a-e01744b89e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:40.519268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c02aa1-802d-479d-ae6a-e01744b89e58 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:40.519545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c02aa1-802d-479d-ae6a-e01744b89e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:40.519822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c02aa1-802d-479d-ae6a-e01744b89e58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:40.520561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c02aa1-802d-479d-ae6a-e01744b89e58 None admin] RBAC: Adding query filter params (role.id=4d98a2b16f8f42bca3e5dac0374cdb54, scope.project.id=955d8cd8cc0f40768c3327e1ced5b79b, user.id=3d90934d388e445184cf04273effcac0) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:40.520871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c02aa1-802d-479d-ae6a-e01744b89e58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:40.522351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c02aa1-802d-479d-ae6a-e01744b89e58 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:40.535449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 97/182] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:40 2022] GET /identity/v3/role_assignments?user.id=3d90934d388e445184cf04273effcac0&scope.project.id=955d8cd8cc0f40768c3327e1ced5b79b&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:40.654255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2e2ef-c29f-49ac-8476-b37af952d398 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:40.767882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2e2ef-c29f-49ac-8476-b37af952d398 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:40.808327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931f869-5843-4d7e-ad23-2c08906c1af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:40.808977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931f869-5843-4d7e-ad23-2c08906c1af3 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:40.809265 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931f869-5843-4d7e-ad23-2c08906c1af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:40.809585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931f869-5843-4d7e-ad23-2c08906c1af3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:40.820089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931f869-5843-4d7e-ad23-2c08906c1af3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-6931f869-5843-4d7e-ad23-2c08906c1af3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:40.826331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:40.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:40.828029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 86/183] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:40.833586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c02aa1-802d-479d-ae6a-e01744b89e58 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:40.856675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6762f5c-bc86-403e-ba86-97d964720a96 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:40.856975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6762f5c-bc86-403e-ba86-97d964720a96 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:40.857229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6762f5c-bc86-403e-ba86-97d964720a96 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:40.858863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 87/184] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:40.865764 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210e69a-de35-48b5-8ac5-baa5f2d873e7 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:40.866050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210e69a-de35-48b5-8ac5-baa5f2d873e7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:40.866326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210e69a-de35-48b5-8ac5-baa5f2d873e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:40.890548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-1210e69a-de35-48b5-8ac5-baa5f2d873e7 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:40.891958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1210e69a-de35-48b5-8ac5-baa5f2d873e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:40.938810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:40.958783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:40.978675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c02aa1-802d-479d-ae6a-e01744b89e58 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:40.999848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 88/185] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:41.018442 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388b95b-8c09-4ee7-a3cb-1f43cc0dc9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:41.019050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388b95b-8c09-4ee7-a3cb-1f43cc0dc9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.019328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388b95b-8c09-4ee7-a3cb-1f43cc0dc9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.019594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388b95b-8c09-4ee7-a3cb-1f43cc0dc9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.020256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388b95b-8c09-4ee7-a3cb-1f43cc0dc9c9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:41.020553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388b95b-8c09-4ee7-a3cb-1f43cc0dc9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:41.021935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388b95b-8c09-4ee7-a3cb-1f43cc0dc9c9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:41.031399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 98/186] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:41.037024 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210e69a-de35-48b5-8ac5-baa5f2d873e7 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:41.143910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210e69a-de35-48b5-8ac5-baa5f2d873e7 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:41.179586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65320af4-cead-47dc-b0b3-2d535259a3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:41.180217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65320af4-cead-47dc-b0b3-2d535259a3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.180492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65320af4-cead-47dc-b0b3-2d535259a3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.181391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65320af4-cead-47dc-b0b3-2d535259a3a6 None admin] PATH_INFO: `/v3/users/89d0a7f4e32441f9a36c06ff5b3e1ae1` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.182621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65320af4-cead-47dc-b0b3-2d535259a3a6 None admin] RBAC: Authorizing `identity:get_user(user_id=89d0a7f4e32441f9a36c06ff5b3e1ae1)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:41.184058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65320af4-cead-47dc-b0b3-2d535259a3a6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:41.186051 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 89/187] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:41 2022] GET /identity/v3/users/89d0a7f4e32441f9a36c06ff5b3e1ae1 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:41.191902 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388b95b-8c09-4ee7-a3cb-1f43cc0dc9c9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:41.262747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d38532-5d66-47af-9083-6a5345acb6d8 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.263493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d38532-5d66-47af-9083-6a5345acb6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.264138 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d38532-5d66-47af-9083-6a5345acb6d8 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.265930 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 90/188] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:29:41.271202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f283d8-9dd3-463c-98a1-0d879accb56e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.271511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f283d8-9dd3-463c-98a1-0d879accb56e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.272241 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f283d8-9dd3-463c-98a1-0d879accb56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.292666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-e5f283d8-9dd3-463c-98a1-0d879accb56e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:41.294049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e5f283d8-9dd3-463c-98a1-0d879accb56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:41.310768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388b95b-8c09-4ee7-a3cb-1f43cc0dc9c9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:41.337655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:41.359659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd4a63-f6e3-46ba-87af-aa0360af907c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:41.360272 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd4a63-f6e3-46ba-87af-aa0360af907c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.360548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd4a63-f6e3-46ba-87af-aa0360af907c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.360817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd4a63-f6e3-46ba-87af-aa0360af907c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.363654 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:41.377108 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 91/189] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:29:41.377850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f283d8-9dd3-463c-98a1-0d879accb56e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:41.446770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd4a63-f6e3-46ba-87af-aa0360af907c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:41.449151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd4a63-f6e3-46ba-87af-aa0360af907c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:41.503102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f283d8-9dd3-463c-98a1-0d879accb56e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:41.543706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af6e6a-84a8-4f3e-b9ad-af1b848c9fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:41.545212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af6e6a-84a8-4f3e-b9ad-af1b848c9fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.545576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af6e6a-84a8-4f3e-b9ad-af1b848c9fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.545936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af6e6a-84a8-4f3e-b9ad-af1b848c9fe8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.559091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af6e6a-84a8-4f3e-b9ad-af1b848c9fe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-16af6e6a-84a8-4f3e-b9ad-af1b848c9fe8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:41.565450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:41.566375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:41.572928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 92/190] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:29:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:41.574502 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af6e6a-84a8-4f3e-b9ad-af1b848c9fe8 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:41.586909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 99/191] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:41.593028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd4a63-f6e3-46ba-87af-aa0360af907c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:41.609915 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:41.628694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:41.692003 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af6e6a-84a8-4f3e-b9ad-af1b848c9fe8 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:41.699907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd4a63-f6e3-46ba-87af-aa0360af907c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:41.728355 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ecd0c-c6ed-423d-adec-9e8ebcbdcd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:41.728977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ecd0c-c6ed-423d-adec-9e8ebcbdcd83 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.729255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ecd0c-c6ed-423d-adec-9e8ebcbdcd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.729528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ecd0c-c6ed-423d-adec-9e8ebcbdcd83 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.730199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ecd0c-c6ed-423d-adec-9e8ebcbdcd83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:41.730540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ecd0c-c6ed-423d-adec-9e8ebcbdcd83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:41.731928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ecd0c-c6ed-423d-adec-9e8ebcbdcd83 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:41.734375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4add16f-66f6-48f5-afa9-5cd87eeffa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:41.735030 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4add16f-66f6-48f5-afa9-5cd87eeffa4f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.735331 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4add16f-66f6-48f5-afa9-5cd87eeffa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.735604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4add16f-66f6-48f5-afa9-5cd87eeffa4f None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.736728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4add16f-66f6-48f5-afa9-5cd87eeffa4f None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:41.738163 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4add16f-66f6-48f5-afa9-5cd87eeffa4f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:41.739810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 100/192] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:41 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:41.746007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 93/193] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:41.746114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb3a01-0750-4736-81c1-86b8c0632349 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.746188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ecd0c-c6ed-423d-adec-9e8ebcbdcd83 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:41.746255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb3a01-0750-4736-81c1-86b8c0632349 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.746255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb3a01-0750-4736-81c1-86b8c0632349 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.746255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 101/194] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:41.746255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb3a01-0750-4736-81c1-86b8c0632349 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:41.838799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:41.847202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ecd0c-c6ed-423d-adec-9e8ebcbdcd83 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:41.848531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb3a01-0750-4736-81c1-86b8c0632349 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:41.850098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:41.884558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c2139-9fdf-488e-892a-e91ae3588889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:41.885507 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c2139-9fdf-488e-892a-e91ae3588889 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.885787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c2139-9fdf-488e-892a-e91ae3588889 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.886057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c2139-9fdf-488e-892a-e91ae3588889 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.887188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c2139-9fdf-488e-892a-e91ae3588889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:41.888388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e12be-59a7-4ca6-9a37-45651b3dc763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:41.888708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c2139-9fdf-488e-892a-e91ae3588889 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:41.889018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e12be-59a7-4ca6-9a37-45651b3dc763 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.889298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e12be-59a7-4ca6-9a37-45651b3dc763 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.889570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e12be-59a7-4ca6-9a37-45651b3dc763 None admin] PATH_INFO: `/v3/users/1351d2cf597543e3baed6c128763d5cd` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.890298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 94/195] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:41.891119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c2139-9fdf-488e-892a-e91ae3588889 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:41.909681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e12be-59a7-4ca6-9a37-45651b3dc763 None admin] RBAC: Authorizing `identity:get_user(user_id=1351d2cf597543e3baed6c128763d5cd)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:41.911194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e12be-59a7-4ca6-9a37-45651b3dc763 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:41.912843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 102/196] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:41 2022] GET /identity/v3/users/1351d2cf597543e3baed6c128763d5cd => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:41.914196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce2bec-6c4d-493e-9bb9-5e1b25c9c035 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:41.914501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce2bec-6c4d-493e-9bb9-5e1b25c9c035 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:41.914771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce2bec-6c4d-493e-9bb9-5e1b25c9c035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:41.934632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1bce2bec-6c4d-493e-9bb9-5e1b25c9c035 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:41.935823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1bce2bec-6c4d-493e-9bb9-5e1b25c9c035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:41.993986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c2139-9fdf-488e-892a-e91ae3588889 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:42.002582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 103/197] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:42.004263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce2bec-6c4d-493e-9bb9-5e1b25c9c035 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:42.029533 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993bbc7-5f56-4d65-abe9-18b5cecb9b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:42.030143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993bbc7-5f56-4d65-abe9-18b5cecb9b43 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:42.030448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993bbc7-5f56-4d65-abe9-18b5cecb9b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:42.030721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993bbc7-5f56-4d65-abe9-18b5cecb9b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:42.031448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993bbc7-5f56-4d65-abe9-18b5cecb9b43 None admin] RBAC: Adding query filter params (role.id=6f434b187a404b7cb1e9bb8303ce59b5, scope.project.id=290b724bdca948a29018c8ecc35d1c8b, user.id=89d0a7f4e32441f9a36c06ff5b3e1ae1) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:42.031760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993bbc7-5f56-4d65-abe9-18b5cecb9b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:42.033206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993bbc7-5f56-4d65-abe9-18b5cecb9b43 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:42.050457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 95/198] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:41 2022] GET /identity/v3/role_assignments?user.id=89d0a7f4e32441f9a36c06ff5b3e1ae1&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=6f434b187a404b7cb1e9bb8303ce59b5 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:42.050457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993bbc7-5f56-4d65-abe9-18b5cecb9b43 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:42.079839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:42.122585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:42.139553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce2bec-6c4d-493e-9bb9-5e1b25c9c035 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:42.185392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54717926-7a35-43a5-b779-6fdaa40ce710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:42.185822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993bbc7-5f56-4d65-abe9-18b5cecb9b43 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:42.186050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54717926-7a35-43a5-b779-6fdaa40ce710 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:42.186334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54717926-7a35-43a5-b779-6fdaa40ce710 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:42.186683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54717926-7a35-43a5-b779-6fdaa40ce710 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:42.187348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54717926-7a35-43a5-b779-6fdaa40ce710 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:42.188918 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54717926-7a35-43a5-b779-6fdaa40ce710 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:42.226288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363723a5-6aea-41d5-8818-a778f2d56bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:42.226988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363723a5-6aea-41d5-8818-a778f2d56bfd None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:42.227328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363723a5-6aea-41d5-8818-a778f2d56bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:42.227636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363723a5-6aea-41d5-8818-a778f2d56bfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:42.301603 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363723a5-6aea-41d5-8818-a778f2d56bfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:42.303034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363723a5-6aea-41d5-8818-a778f2d56bfd None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:42.313950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 104/199] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:29:42 2022] POST /identity/v3/users => generated 297 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:42.314984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54717926-7a35-43a5-b779-6fdaa40ce710 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:42.330788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:42.417124 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 96/200] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:42.422884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363723a5-6aea-41d5-8818-a778f2d56bfd None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:42.445341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54717926-7a35-43a5-b779-6fdaa40ce710 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:42.487184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bd536-9e7b-4afc-91f7-f4bf955d9118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:42.487826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bd536-9e7b-4afc-91f7-f4bf955d9118 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:42.488109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bd536-9e7b-4afc-91f7-f4bf955d9118 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:42.488381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bd536-9e7b-4afc-91f7-f4bf955d9118 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:42.501179 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105bd536-9e7b-4afc-91f7-f4bf955d9118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-105bd536-9e7b-4afc-91f7-f4bf955d9118 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:42.506684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:42.507683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:42.510761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 105/201] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:29:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:42.513693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bd536-9e7b-4afc-91f7-f4bf955d9118 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:42.525119 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:42.551180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363723a5-6aea-41d5-8818-a778f2d56bfd None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:42.581990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:42.595641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4b961-df56-46e7-a2b2-d0486eb25ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:42.596252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4b961-df56-46e7-a2b2-d0486eb25ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:42.596534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4b961-df56-46e7-a2b2-d0486eb25ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:42.596810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4b961-df56-46e7-a2b2-d0486eb25ba6 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:42.598019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4b961-df56-46e7-a2b2-d0486eb25ba6 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:42.600569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4b961-df56-46e7-a2b2-d0486eb25ba6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:42.603244 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 97/202] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:42 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:42.610293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4b961-df56-46e7-a2b2-d0486eb25ba6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:42.642238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bd536-9e7b-4afc-91f7-f4bf955d9118 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:42.687924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9da4c0-8f49-4273-9073-bd9a7775cfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:42.688549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9da4c0-8f49-4273-9073-bd9a7775cfde None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:42.688829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9da4c0-8f49-4273-9073-bd9a7775cfde None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:42.689102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9da4c0-8f49-4273-9073-bd9a7775cfde None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:42.689880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9da4c0-8f49-4273-9073-bd9a7775cfde None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:42.690187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9da4c0-8f49-4273-9073-bd9a7775cfde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:42.691683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9da4c0-8f49-4273-9073-bd9a7775cfde None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:42.700870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 106/203] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:29:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:42.707988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9da4c0-8f49-4273-9073-bd9a7775cfde None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:42.736376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4b961-df56-46e7-a2b2-d0486eb25ba6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:42.775376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3816bab-4da2-45c2-8450-6926d50608ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:42.775988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3816bab-4da2-45c2-8450-6926d50608ed None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:42.776268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3816bab-4da2-45c2-8450-6926d50608ed None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:42.776593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3816bab-4da2-45c2-8450-6926d50608ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:42.777391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3816bab-4da2-45c2-8450-6926d50608ed None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc, user.id=1351d2cf597543e3baed6c128763d5cd) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:42.777749 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3816bab-4da2-45c2-8450-6926d50608ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:42.779362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3816bab-4da2-45c2-8450-6926d50608ed None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:42.785768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 98/204] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:42 2022] GET /identity/v3/role_assignments?user.id=1351d2cf597543e3baed6c128763d5cd&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=d86dba8efe20423abdde06031690beb8 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:42.805183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:42.847568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9da4c0-8f49-4273-9073-bd9a7775cfde None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:42.892459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eee86f-d8c4-46e3-bc30-46e96dcd9234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:42.902949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eee86f-d8c4-46e3-bc30-46e96dcd9234 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:42.902949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eee86f-d8c4-46e3-bc30-46e96dcd9234 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:42.902949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eee86f-d8c4-46e3-bc30-46e96dcd9234 None admin] PATH_INFO: `/v3/users/1351d2cf597543e3baed6c128763d5cd` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:42.903947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eee86f-d8c4-46e3-bc30-46e96dcd9234 None admin] RBAC: Authorizing `identity:get_user(user_id=1351d2cf597543e3baed6c128763d5cd)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:42.905696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eee86f-d8c4-46e3-bc30-46e96dcd9234 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:42.907926 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 107/205] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:42 2022] GET /identity/v3/users/1351d2cf597543e3baed6c128763d5cd => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:42.914046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3816bab-4da2-45c2-8450-6926d50608ed None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:42.980522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:43.029883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3816bab-4da2-45c2-8450-6926d50608ed None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:43.066110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc27db-61fe-4442-8a9b-d900d0d25fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:43.066982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc27db-61fe-4442-8a9b-d900d0d25fea None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:43.067426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc27db-61fe-4442-8a9b-d900d0d25fea None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:43.067831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc27db-61fe-4442-8a9b-d900d0d25fea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:43.141513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc27db-61fe-4442-8a9b-d900d0d25fea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:43.143211 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc27db-61fe-4442-8a9b-d900d0d25fea None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:43.202816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:43.253125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 99/206] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:43.259995 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eee86f-d8c4-46e3-bc30-46e96dcd9234 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:43.381993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eee86f-d8c4-46e3-bc30-46e96dcd9234 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:43.418069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66377be-c53e-4ac8-8ed6-bc15fe621895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:43.418669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66377be-c53e-4ac8-8ed6-bc15fe621895 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:43.418980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66377be-c53e-4ac8-8ed6-bc15fe621895 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:43.419308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66377be-c53e-4ac8-8ed6-bc15fe621895 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:43.420462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66377be-c53e-4ac8-8ed6-bc15fe621895 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:43.421909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66377be-c53e-4ac8-8ed6-bc15fe621895 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:43.423858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 108/207] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:43 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:43.429687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc27db-61fe-4442-8a9b-d900d0d25fea None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:43.550696 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc27db-61fe-4442-8a9b-d900d0d25fea None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:43.589891 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcc9f6-320a-4e90-b7f7-74e71bab1d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:43.590536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcc9f6-320a-4e90-b7f7-74e71bab1d7b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:43.590819 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcc9f6-320a-4e90-b7f7-74e71bab1d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:43.591092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcc9f6-320a-4e90-b7f7-74e71bab1d7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:43.591846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcc9f6-320a-4e90-b7f7-74e71bab1d7b None admin] RBAC: Adding query filter params (role.id=a38face2eeae40c28edda83a4d46614f, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc, user.id=1351d2cf597543e3baed6c128763d5cd) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:43.592157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcc9f6-320a-4e90-b7f7-74e71bab1d7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:43.593758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcc9f6-320a-4e90-b7f7-74e71bab1d7b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:43.600697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 100/208] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:43 2022] GET /identity/v3/role_assignments?user.id=1351d2cf597543e3baed6c128763d5cd&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=a38face2eeae40c28edda83a4d46614f => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:44.480322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23842960-37cb-4041-bd38-cb754d221bbe None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:44.480821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23842960-37cb-4041-bd38-cb754d221bbe None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:44.480893 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23842960-37cb-4041-bd38-cb754d221bbe None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:44.482327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 109/209] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:44.487927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cae71-b07c-410a-a9cf-3139845af04a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:44.488744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cae71-b07c-410a-a9cf-3139845af04a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:44.489220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cae71-b07c-410a-a9cf-3139845af04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:44.533783 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a69ef-8054-4a95-978c-e39fabeac0ad None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:44.534088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a69ef-8054-4a95-978c-e39fabeac0ad None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:44.534331 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a69ef-8054-4a95-978c-e39fabeac0ad None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:44.536055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 110/210] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:29:44.538012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-fb0cae71-b07c-410a-a9cf-3139845af04a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:44.539448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-fb0cae71-b07c-410a-a9cf-3139845af04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:44.541760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad0fa0-c0ea-487b-ae82-26fed2099ba9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:44.542043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad0fa0-c0ea-487b-ae82-26fed2099ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:44.542320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad0fa0-c0ea-487b-ae82-26fed2099ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:44.551286 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:44.559813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:44.567373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c5ad0fa0-c0ea-487b-ae82-26fed2099ba9 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:44.569206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c5ad0fa0-c0ea-487b-ae82-26fed2099ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:44.617554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 101/211] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:44.651205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 111/212] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:44.866143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d00a-34d2-4b21-89e5-8dd5b1a515bf None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:44.866536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d00a-34d2-4b21-89e5-8dd5b1a515bf None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:44.866731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d00a-34d2-4b21-89e5-8dd5b1a515bf None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:44.868909 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 102/213] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:44.874282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b915b-b9ba-429f-b3ad-7bb54ee782c9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:44.874822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b915b-b9ba-429f-b3ad-7bb54ee782c9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:44.875291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b915b-b9ba-429f-b3ad-7bb54ee782c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:44.894022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9d00a-34d2-4b21-89e5-8dd5b1a515bf None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:44.894919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ff6b915b-b9ba-429f-b3ad-7bb54ee782c9 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:44.896331 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ff6b915b-b9ba-429f-b3ad-7bb54ee782c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:44.967984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 112/214] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:44.974587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b915b-b9ba-429f-b3ad-7bb54ee782c9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:45.001793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9d00a-34d2-4b21-89e5-8dd5b1a515bf None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:45.038940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ce565-bc51-4271-9580-b595f900e281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:45.039566 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ce565-bc51-4271-9580-b595f900e281 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:45.039951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ce565-bc51-4271-9580-b595f900e281 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:45.040220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ce565-bc51-4271-9580-b595f900e281 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:45.051387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ce565-bc51-4271-9580-b595f900e281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-7a6ce565-bc51-4271-9580-b595f900e281 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:45.056776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:45.057643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:45.058464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:45.059083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:45.062504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 103/215] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:29:45.064230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ce565-bc51-4271-9580-b595f900e281 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:45.077679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:45.112769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b915b-b9ba-429f-b3ad-7bb54ee782c9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:45.160071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2105d4-ca81-44e2-96a8-b3ae6de7f0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:45.160619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2105d4-ca81-44e2-96a8-b3ae6de7f0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:45.160894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2105d4-ca81-44e2-96a8-b3ae6de7f0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:45.161268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2105d4-ca81-44e2-96a8-b3ae6de7f0a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:45.162486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2105d4-ca81-44e2-96a8-b3ae6de7f0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:45.164041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2105d4-ca81-44e2-96a8-b3ae6de7f0a4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:45.167605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 113/216] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:45.173705 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2105d4-ca81-44e2-96a8-b3ae6de7f0a4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:45.189555 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ce565-bc51-4271-9580-b595f900e281 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:45.242100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a706175-6953-4461-87a3-274842494fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:45.243559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a706175-6953-4461-87a3-274842494fab None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:45.244041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a706175-6953-4461-87a3-274842494fab None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:45.244359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a706175-6953-4461-87a3-274842494fab None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:45.245440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a706175-6953-4461-87a3-274842494fab None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:45.245749 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a706175-6953-4461-87a3-274842494fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:45.247182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a706175-6953-4461-87a3-274842494fab None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:45.256618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 104/217] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:45.257259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a706175-6953-4461-87a3-274842494fab None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:45.286211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2105d4-ca81-44e2-96a8-b3ae6de7f0a4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:45.325083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eb985-e171-46ba-9a2b-fda84dd831a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:45.326450 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:45.326908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb985-e171-46ba-9a2b-fda84dd831a9 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:45.327343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb985-e171-46ba-9a2b-fda84dd831a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:45.327625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb985-e171-46ba-9a2b-fda84dd831a9 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:45.328341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eb985-e171-46ba-9a2b-fda84dd831a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:45.329796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eb985-e171-46ba-9a2b-fda84dd831a9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:45.338209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:45.370253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a706175-6953-4461-87a3-274842494fab None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:45.406094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8c7df-4a77-4e72-b683-ed27bc731551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:45.406770 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8c7df-4a77-4e72-b683-ed27bc731551 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:45.407038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8c7df-4a77-4e72-b683-ed27bc731551 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:45.407308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8c7df-4a77-4e72-b683-ed27bc731551 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:45.417942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8c7df-4a77-4e72-b683-ed27bc731551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-eed8c7df-4a77-4e72-b683-ed27bc731551 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:45.423157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:45.424237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:45.425311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:45.426249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 105/218] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:29:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:45.427155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8c7df-4a77-4e72-b683-ed27bc731551 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:45.457140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 114/219] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:29:45 2022] POST /identity/v3/users => generated 297 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:29:45.457824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eb985-e171-46ba-9a2b-fda84dd831a9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:45.604292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8c7df-4a77-4e72-b683-ed27bc731551 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:45.640752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:45.646524 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e646c17-2a03-4084-aac6-91e79da45fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:45.647425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646c17-2a03-4084-aac6-91e79da45fcf None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:45.647712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646c17-2a03-4084-aac6-91e79da45fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:45.647994 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646c17-2a03-4084-aac6-91e79da45fcf None admin] PATH_INFO: `/v3/users/5665c4e3e54d4031b144eea4d354c2a3` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:45.653783 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eb985-e171-46ba-9a2b-fda84dd831a9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:45.663036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e646c17-2a03-4084-aac6-91e79da45fcf None admin] RBAC: Authorizing `identity:get_user(user_id=5665c4e3e54d4031b144eea4d354c2a3)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:45.666750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e646c17-2a03-4084-aac6-91e79da45fcf None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:45.666970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 106/220] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:45 2022] GET /identity/v3/users/5665c4e3e54d4031b144eea4d354c2a3 => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:45.671385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba1d3e-843f-4249-b3ea-600875f11e60 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:45.671385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba1d3e-843f-4249-b3ea-600875f11e60 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:45.671385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba1d3e-843f-4249-b3ea-600875f11e60 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:45.671385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 107/221] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:45.675437 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba1d3e-843f-4249-b3ea-600875f11e60 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:45.701052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a1a3c-c176-46a2-955f-a0679fa65f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:45.701826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a1a3c-c176-46a2-955f-a0679fa65f99 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:45.702210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a1a3c-c176-46a2-955f-a0679fa65f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:45.702620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a1a3c-c176-46a2-955f-a0679fa65f99 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:45.704550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a1a3c-c176-46a2-955f-a0679fa65f99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:45.704550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a1a3c-c176-46a2-955f-a0679fa65f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:45.705042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a1a3c-c176-46a2-955f-a0679fa65f99 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:45.716952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 115/222] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:45.718190 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b84a8a-e631-4a36-973a-0b1fdb9afb7a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:45.718498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b84a8a-e631-4a36-973a-0b1fdb9afb7a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:45.718775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b84a8a-e631-4a36-973a-0b1fdb9afb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:45.739099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-35b84a8a-e631-4a36-973a-0b1fdb9afb7a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:45.740330 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-35b84a8a-e631-4a36-973a-0b1fdb9afb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:45.804095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:45.820218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba1d3e-843f-4249-b3ea-600875f11e60 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:45.832313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 116/223] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:45.833456 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b84a8a-e631-4a36-973a-0b1fdb9afb7a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:45.871283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc208159-a8de-44f0-bc6a-48f341e8de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:45.872353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc208159-a8de-44f0-bc6a-48f341e8de31 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:45.872844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc208159-a8de-44f0-bc6a-48f341e8de31 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:45.873307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc208159-a8de-44f0-bc6a-48f341e8de31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:45.986024 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:45.996529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc208159-a8de-44f0-bc6a-48f341e8de31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:45.998180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc208159-a8de-44f0-bc6a-48f341e8de31 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:46.022556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b84a8a-e631-4a36-973a-0b1fdb9afb7a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:46.093005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43394790-ba43-4ffa-8fdb-8d28a98b078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:46.093645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43394790-ba43-4ffa-8fdb-8d28a98b078c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:46.093918 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43394790-ba43-4ffa-8fdb-8d28a98b078c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:46.094187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43394790-ba43-4ffa-8fdb-8d28a98b078c None admin] PATH_INFO: `/v3/users/1351d2cf597543e3baed6c128763d5cd` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:46.096145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43394790-ba43-4ffa-8fdb-8d28a98b078c None admin] RBAC: Authorizing `identity:get_user(user_id=1351d2cf597543e3baed6c128763d5cd)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:46.097610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43394790-ba43-4ffa-8fdb-8d28a98b078c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:46.100498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 117/224] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:45 2022] GET /identity/v3/users/1351d2cf597543e3baed6c128763d5cd => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:46.106884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43394790-ba43-4ffa-8fdb-8d28a98b078c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:46.153663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 108/225] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:46.155062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc208159-a8de-44f0-bc6a-48f341e8de31 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:46.159665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:46.212310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43394790-ba43-4ffa-8fdb-8d28a98b078c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:46.247916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cab268-b9a0-4a00-941d-2f15e5549f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:46.248284 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab268-b9a0-4a00-941d-2f15e5549f90 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:46.248558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab268-b9a0-4a00-941d-2f15e5549f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:46.248823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab268-b9a0-4a00-941d-2f15e5549f90 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:46.249954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cab268-b9a0-4a00-941d-2f15e5549f90 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:46.251432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cab268-b9a0-4a00-941d-2f15e5549f90 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:46.253066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 118/226] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:46 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:46.258092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cab268-b9a0-4a00-941d-2f15e5549f90 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:46.266795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:46.273784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc208159-a8de-44f0-bc6a-48f341e8de31 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:46.311249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f02610-ab6e-4de0-9b30-4aff37d72868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:46.311876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f02610-ab6e-4de0-9b30-4aff37d72868 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:46.312147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f02610-ab6e-4de0-9b30-4aff37d72868 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:46.312713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f02610-ab6e-4de0-9b30-4aff37d72868 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:46.328011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f02610-ab6e-4de0-9b30-4aff37d72868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-b2f02610-ab6e-4de0-9b30-4aff37d72868 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:46.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:46.334020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:46.334867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 109/227] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:29:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:46.335590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f02610-ab6e-4de0-9b30-4aff37d72868 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:46.397466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cab268-b9a0-4a00-941d-2f15e5549f90 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:46.420511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:46.445521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55794dd-9228-4297-b014-89dc026b2626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:46.446345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55794dd-9228-4297-b014-89dc026b2626 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:46.446724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55794dd-9228-4297-b014-89dc026b2626 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:46.447060 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55794dd-9228-4297-b014-89dc026b2626 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:46.447181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f02610-ab6e-4de0-9b30-4aff37d72868 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:46.448336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55794dd-9228-4297-b014-89dc026b2626 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:46.450941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55794dd-9228-4297-b014-89dc026b2626 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:46.452971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 119/228] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:46 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:46.453842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55794dd-9228-4297-b014-89dc026b2626 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:46.483923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26419c65-d3a1-47b2-97ab-3466eb986280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:46.484732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26419c65-d3a1-47b2-97ab-3466eb986280 None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:46.485055 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26419c65-d3a1-47b2-97ab-3466eb986280 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:46.485437 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26419c65-d3a1-47b2-97ab-3466eb986280 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/users/1351d2cf597543e3baed6c128763d5cd/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:46.487517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26419c65-d3a1-47b2-97ab-3466eb986280 None admin] RBAC: Authorizing `identity:create_grant(project_id=45aa2f6db17f4df1b7f892a95d31b8fc, user_id=1351d2cf597543e3baed6c128763d5cd, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:46.489147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26419c65-d3a1-47b2-97ab-3466eb986280 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:46.563807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55794dd-9228-4297-b014-89dc026b2626 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:46.572630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 110/229] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:29:46 2022] PUT /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/users/1351d2cf597543e3baed6c128763d5cd/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:29:46.574102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26419c65-d3a1-47b2-97ab-3466eb986280 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:46.606799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:46.645833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84f244-b08e-4e7f-a4db-661d214a92f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:46.646470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84f244-b08e-4e7f-a4db-661d214a92f2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:46.646750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84f244-b08e-4e7f-a4db-661d214a92f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:46.647020 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84f244-b08e-4e7f-a4db-661d214a92f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:46.649532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84f244-b08e-4e7f-a4db-661d214a92f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:46.649805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84f244-b08e-4e7f-a4db-661d214a92f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:46.654812 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84f244-b08e-4e7f-a4db-661d214a92f2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:46.674553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 120/230] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:29:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:46.678192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84f244-b08e-4e7f-a4db-661d214a92f2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:46.769385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:46.790316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26419c65-d3a1-47b2-97ab-3466eb986280 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:46.837380 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84f244-b08e-4e7f-a4db-661d214a92f2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:46.839847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d0baa-cb83-47f5-9da7-9d61e772e403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:46.840727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d0baa-cb83-47f5-9da7-9d61e772e403 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:46.842114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d0baa-cb83-47f5-9da7-9d61e772e403 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:46.842596 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d0baa-cb83-47f5-9da7-9d61e772e403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:46.843598 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d0baa-cb83-47f5-9da7-9d61e772e403 None admin] RBAC: Adding query filter params (role.id=4d98a2b16f8f42bca3e5dac0374cdb54, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc, user.id=5665c4e3e54d4031b144eea4d354c2a3) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:46.844099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d0baa-cb83-47f5-9da7-9d61e772e403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:46.845775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d0baa-cb83-47f5-9da7-9d61e772e403 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:46.857673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 111/231] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:46 2022] GET /identity/v3/role_assignments?user.id=5665c4e3e54d4031b144eea4d354c2a3&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:46.896887 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0c289-f4cf-4af0-8bf3-c65f685aaaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:46.897807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0c289-f4cf-4af0-8bf3-c65f685aaaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:46.898142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0c289-f4cf-4af0-8bf3-c65f685aaaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:46.898414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0c289-f4cf-4af0-8bf3-c65f685aaaf4 None admin] PATH_INFO: `/v3/users/1351d2cf597543e3baed6c128763d5cd` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:46.901443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0c289-f4cf-4af0-8bf3-c65f685aaaf4 None admin] RBAC: Authorizing `identity:get_user(user_id=1351d2cf597543e3baed6c128763d5cd)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:46.902977 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0c289-f4cf-4af0-8bf3-c65f685aaaf4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:46.905071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 121/232] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:46 2022] GET /identity/v3/users/1351d2cf597543e3baed6c128763d5cd => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:46.910550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d0baa-cb83-47f5-9da7-9d61e772e403 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:47.014604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:47.046472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d0baa-cb83-47f5-9da7-9d61e772e403 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:47.087343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf05de-eee6-4310-a221-85a625ca82fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:47.088401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf05de-eee6-4310-a221-85a625ca82fc None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:47.088883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf05de-eee6-4310-a221-85a625ca82fc None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:47.089354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf05de-eee6-4310-a221-85a625ca82fc None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:47.090998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf05de-eee6-4310-a221-85a625ca82fc None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:47.092654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf05de-eee6-4310-a221-85a625ca82fc None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:47.094662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 112/233] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:46 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:47.100667 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0c289-f4cf-4af0-8bf3-c65f685aaaf4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:47.228907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0c289-f4cf-4af0-8bf3-c65f685aaaf4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:47.247627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:47.272484 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c2085-3487-4c5e-b5a2-a95089a16294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:47.273196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c2085-3487-4c5e-b5a2-a95089a16294 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:47.273471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c2085-3487-4c5e-b5a2-a95089a16294 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:47.274334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c2085-3487-4c5e-b5a2-a95089a16294 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/users/1351d2cf597543e3baed6c128763d5cd/roles/a38face2eeae40c28edda83a4d46614f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:47.276642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c2085-3487-4c5e-b5a2-a95089a16294 None admin] RBAC: Authorizing `identity:create_grant(project_id=45aa2f6db17f4df1b7f892a95d31b8fc, user_id=1351d2cf597543e3baed6c128763d5cd, role_id=a38face2eeae40c28edda83a4d46614f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:47.278235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c2085-3487-4c5e-b5a2-a95089a16294 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:47.383975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 122/234] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:29:47 2022] PUT /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/users/1351d2cf597543e3baed6c128763d5cd/roles/a38face2eeae40c28edda83a4d46614f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:29:47.738610 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a6c32-29ef-4fa8-8a14-94b961554b51 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:47.739428 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a6c32-29ef-4fa8-8a14-94b961554b51 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:47.739850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a6c32-29ef-4fa8-8a14-94b961554b51 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:47.741358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 113/235] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:47.755246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc53ab-bef9-4b79-9290-58b8ec8a3bf1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:47.755718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc53ab-bef9-4b79-9290-58b8ec8a3bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:47.755791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc53ab-bef9-4b79-9290-58b8ec8a3bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:47.779776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1bbc53ab-bef9-4b79-9290-58b8ec8a3bf1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:47.780979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1bbc53ab-bef9-4b79-9290-58b8ec8a3bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:47.869654 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 123/236] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:47.987853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef42dd-cff1-485d-a51e-b1ad479d3082 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:47.988374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef42dd-cff1-485d-a51e-b1ad479d3082 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:47.988700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef42dd-cff1-485d-a51e-b1ad479d3082 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:47.990144 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 114/237] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:47.996243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc390-6315-4e4a-95f4-1103de782f72 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:47.996530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc390-6315-4e4a-95f4-1103de782f72 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:47.996811 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc390-6315-4e4a-95f4-1103de782f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:48.019969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-649cc390-6315-4e4a-95f4-1103de782f72 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:48.021218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-649cc390-6315-4e4a-95f4-1103de782f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:48.113195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 124/238] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:48.328101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef42dd-cff1-485d-a51e-b1ad479d3082 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:48.483105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef42dd-cff1-485d-a51e-b1ad479d3082 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:48.512343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cc390-6315-4e4a-95f4-1103de782f72 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:48.539711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddea59-c765-424d-aaaa-8b7b19b58962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:48.541490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddea59-c765-424d-aaaa-8b7b19b58962 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:48.541776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddea59-c765-424d-aaaa-8b7b19b58962 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:48.542047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddea59-c765-424d-aaaa-8b7b19b58962 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:48.564717 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddea59-c765-424d-aaaa-8b7b19b58962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-f5ddea59-c765-424d-aaaa-8b7b19b58962 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:48.575631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:48.576532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:48.577333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:48.577925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:48.578004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 115/239] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:48.588302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddea59-c765-424d-aaaa-8b7b19b58962 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:48.592893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:48.699055 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddea59-c765-424d-aaaa-8b7b19b58962 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:48.736899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f1162-24a4-4d66-82d6-68890948f7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:48.737565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1162-24a4-4d66-82d6-68890948f7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:48.737847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1162-24a4-4d66-82d6-68890948f7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:48.738120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1162-24a4-4d66-82d6-68890948f7f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:48.738869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f1162-24a4-4d66-82d6-68890948f7f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:48.739501 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f1162-24a4-4d66-82d6-68890948f7f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:48.741048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f1162-24a4-4d66-82d6-68890948f7f2 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:48.750237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 116/240] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:48.755637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f1162-24a4-4d66-82d6-68890948f7f2 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:48.766534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cc390-6315-4e4a-95f4-1103de782f72 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:48.828594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:48.839663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fc8cf-fdae-4a72-8abd-6da484adcd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:48.839934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fc8cf-fdae-4a72-8abd-6da484adcd37 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:48.840189 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fc8cf-fdae-4a72-8abd-6da484adcd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:48.840461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fc8cf-fdae-4a72-8abd-6da484adcd37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:48.841647 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fc8cf-fdae-4a72-8abd-6da484adcd37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:48.843199 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fc8cf-fdae-4a72-8abd-6da484adcd37 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:48.845147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 125/241] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:48.850388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fc8cf-fdae-4a72-8abd-6da484adcd37 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:48.871638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f1162-24a4-4d66-82d6-68890948f7f2 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:48.911234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85c08e-ec80-4380-aab2-3b8c7f0ae54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:48.911923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85c08e-ec80-4380-aab2-3b8c7f0ae54b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:48.912210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85c08e-ec80-4380-aab2-3b8c7f0ae54b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:48.912480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85c08e-ec80-4380-aab2-3b8c7f0ae54b None admin] PATH_INFO: `/v3/users/d55e95742e7c43e38a9d970fddcd335b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:48.928704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85c08e-ec80-4380-aab2-3b8c7f0ae54b None admin] RBAC: Authorizing `identity:get_user(user_id=d55e95742e7c43e38a9d970fddcd335b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:48.930514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85c08e-ec80-4380-aab2-3b8c7f0ae54b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:48.933325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 117/242] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:48 2022] GET /identity/v3/users/d55e95742e7c43e38a9d970fddcd335b => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:48.939329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85c08e-ec80-4380-aab2-3b8c7f0ae54b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:49.045609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fc8cf-fdae-4a72-8abd-6da484adcd37 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:49.064560 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85c08e-ec80-4380-aab2-3b8c7f0ae54b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:49.071295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:49.082026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f218e-4ec1-40a9-903f-69fc017f5eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:49.083049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f218e-4ec1-40a9-903f-69fc017f5eab None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:49.083522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f218e-4ec1-40a9-903f-69fc017f5eab None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:49.084014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f218e-4ec1-40a9-903f-69fc017f5eab None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:49.084867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f218e-4ec1-40a9-903f-69fc017f5eab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:49.086702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f218e-4ec1-40a9-903f-69fc017f5eab None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:49.093662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:49.108609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60c8f7-971d-45b4-bba9-56b275fd9c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:49.109219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60c8f7-971d-45b4-bba9-56b275fd9c07 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:49.109488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60c8f7-971d-45b4-bba9-56b275fd9c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:49.109781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60c8f7-971d-45b4-bba9-56b275fd9c07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:49.174543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 126/243] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:29:48 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:29:49.176015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae9722-cc1b-47c5-b5b7-333717dddbdd None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:49.176309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae9722-cc1b-47c5-b5b7-333717dddbdd None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:49.176632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae9722-cc1b-47c5-b5b7-333717dddbdd None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:49.178227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 127/244] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:49.186388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b1274-73ee-4ba7-b647-ee3e7ed054df None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:49.186689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b1274-73ee-4ba7-b647-ee3e7ed054df None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:49.188740 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b1274-73ee-4ba7-b647-ee3e7ed054df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:49.192055 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60c8f7-971d-45b4-bba9-56b275fd9c07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:49.193909 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60c8f7-971d-45b4-bba9-56b275fd9c07 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:49.229641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3e9b1274-73ee-4ba7-b647-ee3e7ed054df None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:49.230878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3e9b1274-73ee-4ba7-b647-ee3e7ed054df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:49.327555 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 118/245] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 23:29:49.334007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60c8f7-971d-45b4-bba9-56b275fd9c07 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:49.387567 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:49.401243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 128/246] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:49.484203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60c8f7-971d-45b4-bba9-56b275fd9c07 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:49.510999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188b0e4-5dcc-424c-8408-d36dc444fde8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:49.511330 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188b0e4-5dcc-424c-8408-d36dc444fde8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:49.511609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188b0e4-5dcc-424c-8408-d36dc444fde8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:49.513175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 129/247] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:49.519723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429b7fb-857b-4f45-aed3-ddff44b68926 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:49.520002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429b7fb-857b-4f45-aed3-ddff44b68926 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:49.520274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429b7fb-857b-4f45-aed3-ddff44b68926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:49.524515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807c121-cf56-450d-a74a-61dd211087d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:49.525292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807c121-cf56-450d-a74a-61dd211087d3 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:49.525621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807c121-cf56-450d-a74a-61dd211087d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:49.525900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807c121-cf56-450d-a74a-61dd211087d3 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:49.527239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807c121-cf56-450d-a74a-61dd211087d3 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:49.528788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807c121-cf56-450d-a74a-61dd211087d3 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:49.531837 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 119/248] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:49 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:49.538215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807c121-cf56-450d-a74a-61dd211087d3 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:49.549386 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5429b7fb-857b-4f45-aed3-ddff44b68926 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:49.555656 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5429b7fb-857b-4f45-aed3-ddff44b68926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:49.659069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 130/249] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:49.659230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:49.660750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509277f5-21d3-4051-9490-d6ad43bf2125 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:49.661180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509277f5-21d3-4051-9490-d6ad43bf2125 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:49.661483 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509277f5-21d3-4051-9490-d6ad43bf2125 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:49.663085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 131/250] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:49.669018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1fc22-8464-480b-a854-eae744a63f2e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:49.669414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1fc22-8464-480b-a854-eae744a63f2e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:49.669994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1fc22-8464-480b-a854-eae744a63f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:49.672913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807c121-cf56-450d-a74a-61dd211087d3 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:49.702905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0aa1fc22-8464-480b-a854-eae744a63f2e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:49.704217 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0aa1fc22-8464-480b-a854-eae744a63f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:49.712098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39a79e-5682-4f7f-bb3e-aec7af171142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:49.712931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39a79e-5682-4f7f-bb3e-aec7af171142 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:49.713375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39a79e-5682-4f7f-bb3e-aec7af171142 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:49.713791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39a79e-5682-4f7f-bb3e-aec7af171142 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:49.714698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39a79e-5682-4f7f-bb3e-aec7af171142 None admin] RBAC: Adding query filter params (role.id=6f434b187a404b7cb1e9bb8303ce59b5, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc, user.id=d55e95742e7c43e38a9d970fddcd335b) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:49.715142 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39a79e-5682-4f7f-bb3e-aec7af171142 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:49.716890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39a79e-5682-4f7f-bb3e-aec7af171142 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:49.723046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 120/251] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:49 2022] GET /identity/v3/role_assignments?user.id=d55e95742e7c43e38a9d970fddcd335b&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=6f434b187a404b7cb1e9bb8303ce59b5 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:49.724031 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39a79e-5682-4f7f-bb3e-aec7af171142 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:49.724396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:49.802757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 132/252] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:29:49.852727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39a79e-5682-4f7f-bb3e-aec7af171142 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:49.891625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68b3cf-d990-4f88-aa11-2cea523cac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:49.892642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68b3cf-d990-4f88-aa11-2cea523cac87 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:49.893145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68b3cf-d990-4f88-aa11-2cea523cac87 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:49.893589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68b3cf-d990-4f88-aa11-2cea523cac87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:49.907146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68b3cf-d990-4f88-aa11-2cea523cac87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-1b68b3cf-d990-4f88-aa11-2cea523cac87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:49.912433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:49.913314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:29:49.914736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:49.917068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 121/253] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:29:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:49.923679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1fc22-8464-480b-a854-eae744a63f2e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:50.009976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68b3cf-d990-4f88-aa11-2cea523cac87 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:50.034641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:50.043220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1fc22-8464-480b-a854-eae744a63f2e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:50.079548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab6a87-3b26-4b78-bbf9-8ce53c5abce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:50.080155 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab6a87-3b26-4b78-bbf9-8ce53c5abce3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:50.080430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab6a87-3b26-4b78-bbf9-8ce53c5abce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:50.080705 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab6a87-3b26-4b78-bbf9-8ce53c5abce3 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:50.081374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab6a87-3b26-4b78-bbf9-8ce53c5abce3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:50.081673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab6a87-3b26-4b78-bbf9-8ce53c5abce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:50.083122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab6a87-3b26-4b78-bbf9-8ce53c5abce3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:50.093338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 133/254] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:50.101950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab6a87-3b26-4b78-bbf9-8ce53c5abce3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:50.122811 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:50.188614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68b3cf-d990-4f88-aa11-2cea523cac87 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:50.229356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f909f47-b662-40ea-869e-b9bb71c433d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:50.230298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f909f47-b662-40ea-869e-b9bb71c433d1 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:50.230784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f909f47-b662-40ea-869e-b9bb71c433d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:50.231401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f909f47-b662-40ea-869e-b9bb71c433d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:50.254835 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f909f47-b662-40ea-869e-b9bb71c433d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-1f909f47-b662-40ea-869e-b9bb71c433d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:50.260329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:50.261218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:50.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:50.266659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 122/255] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:29:50.268892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f909f47-b662-40ea-869e-b9bb71c433d1 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:50.276510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab6a87-3b26-4b78-bbf9-8ce53c5abce3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:50.341755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9d240-2db1-4030-bc3f-cde58b9bf1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:50.342657 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9d240-2db1-4030-bc3f-cde58b9bf1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:50.343963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9d240-2db1-4030-bc3f-cde58b9bf1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:50.344380 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9d240-2db1-4030-bc3f-cde58b9bf1d3 None admin] PATH_INFO: `/v3/users/1351d2cf597543e3baed6c128763d5cd` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:50.346175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9d240-2db1-4030-bc3f-cde58b9bf1d3 None admin] RBAC: Authorizing `identity:get_user(user_id=1351d2cf597543e3baed6c128763d5cd)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:50.348656 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9d240-2db1-4030-bc3f-cde58b9bf1d3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:50.354724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 134/256] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:50 2022] GET /identity/v3/users/1351d2cf597543e3baed6c128763d5cd => generated 283 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:50.354724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9d240-2db1-4030-bc3f-cde58b9bf1d3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:50.428894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:50.490900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:50.492448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f909f47-b662-40ea-869e-b9bb71c433d1 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:50.517278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9d240-2db1-4030-bc3f-cde58b9bf1d3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:50.565423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfda19-e2fe-4cb0-9730-8fd548821702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:50.566054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfda19-e2fe-4cb0-9730-8fd548821702 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:50.566331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfda19-e2fe-4cb0-9730-8fd548821702 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:50.566625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfda19-e2fe-4cb0-9730-8fd548821702 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:50.571953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a488e-16dd-43a2-a1b2-cd48c56b70dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:50.572564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a488e-16dd-43a2-a1b2-cd48c56b70dd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:50.572844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a488e-16dd-43a2-a1b2-cd48c56b70dd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:50.573113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a488e-16dd-43a2-a1b2-cd48c56b70dd None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:50.573782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a488e-16dd-43a2-a1b2-cd48c56b70dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:50.574085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a488e-16dd-43a2-a1b2-cd48c56b70dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:50.575501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a488e-16dd-43a2-a1b2-cd48c56b70dd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:50.602626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 135/257] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:50.603570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a488e-16dd-43a2-a1b2-cd48c56b70dd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:50.607854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfda19-e2fe-4cb0-9730-8fd548821702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-dedfda19-e2fe-4cb0-9730-8fd548821702 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:50.622905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:50.624118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 123/258] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:29:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:50.624888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfda19-e2fe-4cb0-9730-8fd548821702 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:50.738737 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a488e-16dd-43a2-a1b2-cd48c56b70dd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:50.746793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfda19-e2fe-4cb0-9730-8fd548821702 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:50.784036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7b03e-ecd4-4eb4-9bc1-2655ee532635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:50.784665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7b03e-ecd4-4eb4-9bc1-2655ee532635 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:50.784944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7b03e-ecd4-4eb4-9bc1-2655ee532635 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:50.785218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7b03e-ecd4-4eb4-9bc1-2655ee532635 None admin] PATH_INFO: `/v3/users/5665c4e3e54d4031b144eea4d354c2a3` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:50.786410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7b03e-ecd4-4eb4-9bc1-2655ee532635 None admin] RBAC: Authorizing `identity:get_user(user_id=5665c4e3e54d4031b144eea4d354c2a3)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:50.787995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7b03e-ecd4-4eb4-9bc1-2655ee532635 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:50.788183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec38a2-77b9-47b6-9600-8fd06b93e3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:50.788621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec38a2-77b9-47b6-9600-8fd06b93e3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:50.788901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec38a2-77b9-47b6-9600-8fd06b93e3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:50.789374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec38a2-77b9-47b6-9600-8fd06b93e3a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:50.790033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 124/259] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:50 2022] GET /identity/v3/users/5665c4e3e54d4031b144eea4d354c2a3 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:50.790612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7b03e-ecd4-4eb4-9bc1-2655ee532635 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:50.793280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:50.811250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:50.901804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7b03e-ecd4-4eb4-9bc1-2655ee532635 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:50.938294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4324930-2e65-426b-b9ae-cb135ebe8452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:50.938937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4324930-2e65-426b-b9ae-cb135ebe8452 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:50.939221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4324930-2e65-426b-b9ae-cb135ebe8452 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:50.939496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4324930-2e65-426b-b9ae-cb135ebe8452 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:50.940227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4324930-2e65-426b-b9ae-cb135ebe8452 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:50.940538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4324930-2e65-426b-b9ae-cb135ebe8452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:50.942110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4324930-2e65-426b-b9ae-cb135ebe8452 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:50.951007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 125/260] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:29:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:50.951916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4324930-2e65-426b-b9ae-cb135ebe8452 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:50.967274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec38a2-77b9-47b6-9600-8fd06b93e3a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:50.968809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec38a2-77b9-47b6-9600-8fd06b93e3a6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:51.022063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:51.064527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4324930-2e65-426b-b9ae-cb135ebe8452 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:51.102882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df0069-4a86-4c16-b3af-203be5547cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:51.103671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df0069-4a86-4c16-b3af-203be5547cfa None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:51.103920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df0069-4a86-4c16-b3af-203be5547cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:51.104194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df0069-4a86-4c16-b3af-203be5547cfa None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:51.105365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df0069-4a86-4c16-b3af-203be5547cfa None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:51.106970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df0069-4a86-4c16-b3af-203be5547cfa None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:51.108789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 126/261] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:50 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:51.109388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df0069-4a86-4c16-b3af-203be5547cfa None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:51.213924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 136/262] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:51.215143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec38a2-77b9-47b6-9600-8fd06b93e3a6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:51.216829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df0069-4a86-4c16-b3af-203be5547cfa None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:51.254391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcadb19-3876-48aa-88fe-0a31ab7ef1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:51.255168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcadb19-3876-48aa-88fe-0a31ab7ef1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:51.255441 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcadb19-3876-48aa-88fe-0a31ab7ef1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:51.255726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcadb19-3876-48aa-88fe-0a31ab7ef1e6 None admin] PATH_INFO: `/v3/users/1351d2cf597543e3baed6c128763d5cd` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:51.257014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcadb19-3876-48aa-88fe-0a31ab7ef1e6 None admin] RBAC: Authorizing `identity:get_user(user_id=1351d2cf597543e3baed6c128763d5cd)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:51.259362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcadb19-3876-48aa-88fe-0a31ab7ef1e6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:51.261079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 127/263] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:51 2022] GET /identity/v3/users/1351d2cf597543e3baed6c128763d5cd => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:51.261863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcadb19-3876-48aa-88fe-0a31ab7ef1e6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:51.265663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:51.285909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:51.390930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec38a2-77b9-47b6-9600-8fd06b93e3a6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:51.419216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcadb19-3876-48aa-88fe-0a31ab7ef1e6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:51.464241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb89eb9-5ec5-46a8-935b-915f0d73c5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:51.465828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb89eb9-5ec5-46a8-935b-915f0d73c5f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:51.466320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb89eb9-5ec5-46a8-935b-915f0d73c5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:51.467127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb5de3-0b1b-48e1-b34d-2ba79beb8765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:51.467962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb5de3-0b1b-48e1-b34d-2ba79beb8765 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:51.468449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb89eb9-5ec5-46a8-935b-915f0d73c5f5 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/users/5665c4e3e54d4031b144eea4d354c2a3/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:51.468933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb5de3-0b1b-48e1-b34d-2ba79beb8765 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:51.470200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb5de3-0b1b-48e1-b34d-2ba79beb8765 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:51.471989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb5de3-0b1b-48e1-b34d-2ba79beb8765 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:51.472815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb89eb9-5ec5-46a8-935b-915f0d73c5f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=45aa2f6db17f4df1b7f892a95d31b8fc, user_id=5665c4e3e54d4031b144eea4d354c2a3, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:51.474453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb5de3-0b1b-48e1-b34d-2ba79beb8765 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:51.477175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb89eb9-5ec5-46a8-935b-915f0d73c5f5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:51.479839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 128/264] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:51 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:51.482036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb5de3-0b1b-48e1-b34d-2ba79beb8765 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:51.552106 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:51.588792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb5de3-0b1b-48e1-b34d-2ba79beb8765 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:51.625455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d3978-2e41-4f69-b1b9-11000b27e0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:51.628087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3978-2e41-4f69-b1b9-11000b27e0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:51.630666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3978-2e41-4f69-b1b9-11000b27e0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:51.632778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 137/265] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:29:51 2022] PUT /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/users/5665c4e3e54d4031b144eea4d354c2a3/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:29:51.633616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb89eb9-5ec5-46a8-935b-915f0d73c5f5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:51.634264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3978-2e41-4f69-b1b9-11000b27e0d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:51.714986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d3978-2e41-4f69-b1b9-11000b27e0d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:51.716688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d3978-2e41-4f69-b1b9-11000b27e0d6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:51.794360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:51.832931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 129/266] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:51.839681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d3978-2e41-4f69-b1b9-11000b27e0d6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:51.866399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:51.869270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb89eb9-5ec5-46a8-935b-915f0d73c5f5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:51.910202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef2f4f-73ad-4003-b5cb-31223f06ef52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:51.910994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef2f4f-73ad-4003-b5cb-31223f06ef52 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:51.911275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef2f4f-73ad-4003-b5cb-31223f06ef52 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:51.911544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef2f4f-73ad-4003-b5cb-31223f06ef52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:51.912279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef2f4f-73ad-4003-b5cb-31223f06ef52 None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc, user.id=1351d2cf597543e3baed6c128763d5cd) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:51.912585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef2f4f-73ad-4003-b5cb-31223f06ef52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:51.915325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef2f4f-73ad-4003-b5cb-31223f06ef52 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:51.922827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 138/267] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:51 2022] GET /identity/v3/role_assignments?user.id=1351d2cf597543e3baed6c128763d5cd&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=d86dba8efe20423abdde06031690beb8 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:52.012290 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d3978-2e41-4f69-b1b9-11000b27e0d6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:52.069152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fc318-b9c6-4c5c-969a-8e2bfb15188f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:52.070612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fc318-b9c6-4c5c-969a-8e2bfb15188f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:52.071175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fc318-b9c6-4c5c-969a-8e2bfb15188f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:52.071598 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fc318-b9c6-4c5c-969a-8e2bfb15188f None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:52.073052 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fc318-b9c6-4c5c-969a-8e2bfb15188f None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:52.074707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fc318-b9c6-4c5c-969a-8e2bfb15188f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:52.076812 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 130/268] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:51 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:52.082735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef2f4f-73ad-4003-b5cb-31223f06ef52 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:52.163724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99500787-ec0b-4617-8fdc-6fdce7becffc None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:52.164620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99500787-ec0b-4617-8fdc-6fdce7becffc None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:52.164832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99500787-ec0b-4617-8fdc-6fdce7becffc None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:52.167614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 131/269] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:29:52.172011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414d9c2-67f7-4fc7-87fe-bca535d91e1d None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:52.172291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414d9c2-67f7-4fc7-87fe-bca535d91e1d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:52.172564 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414d9c2-67f7-4fc7-87fe-bca535d91e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:52.192719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-5414d9c2-67f7-4fc7-87fe-bca535d91e1d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:52.193967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5414d9c2-67f7-4fc7-87fe-bca535d91e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:52.248883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:52.252210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef2f4f-73ad-4003-b5cb-31223f06ef52 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:52.332541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564544c-8da4-4085-8fa6-9bde32bc4da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:52.333150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564544c-8da4-4085-8fa6-9bde32bc4da3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:52.333423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564544c-8da4-4085-8fa6-9bde32bc4da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:52.333692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564544c-8da4-4085-8fa6-9bde32bc4da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:52.334438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564544c-8da4-4085-8fa6-9bde32bc4da3 None admin] RBAC: Adding query filter params (role.id=a38face2eeae40c28edda83a4d46614f, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc, user.id=1351d2cf597543e3baed6c128763d5cd) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:52.334992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564544c-8da4-4085-8fa6-9bde32bc4da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:52.338606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564544c-8da4-4085-8fa6-9bde32bc4da3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:52.341286 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 132/270] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:52.350526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 139/271] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:52 2022] GET /identity/v3/role_assignments?user.id=1351d2cf597543e3baed6c128763d5cd&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=a38face2eeae40c28edda83a4d46614f => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:52.395335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21e505-7b3d-4878-96f4-efff3a9f86df None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:52.395637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21e505-7b3d-4878-96f4-efff3a9f86df None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:52.395884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21e505-7b3d-4878-96f4-efff3a9f86df None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:52.398074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 133/272] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:52.404257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916e1c1-089b-4ff9-b8b1-c75167dd291e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:52.404618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916e1c1-089b-4ff9-b8b1-c75167dd291e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:52.404815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916e1c1-089b-4ff9-b8b1-c75167dd291e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:52.430294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b916e1c1-089b-4ff9-b8b1-c75167dd291e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:52.431584 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b916e1c1-089b-4ff9-b8b1-c75167dd291e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:52.463174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:52.512731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 140/273] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:52.585460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e6cd3-c4a7-4bb4-9f80-b20ffa4fe035 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:52.585821 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e6cd3-c4a7-4bb4-9f80-b20ffa4fe035 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:52.586244 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e6cd3-c4a7-4bb4-9f80-b20ffa4fe035 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:52.587917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 134/274] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:52.600550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03540c6c-59bc-4697-a941-9a4568053ce1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:52.600853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03540c6c-59bc-4697-a941-9a4568053ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:52.601098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03540c6c-59bc-4697-a941-9a4568053ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:52.625879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-03540c6c-59bc-4697-a941-9a4568053ce1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:52.627127 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-03540c6c-59bc-4697-a941-9a4568053ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:52.681307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e6cd3-c4a7-4bb4-9f80-b20ffa4fe035 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:52.728685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 141/275] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:52.814588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e6cd3-c4a7-4bb4-9f80-b20ffa4fe035 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:52.830104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03540c6c-59bc-4697-a941-9a4568053ce1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:52.851194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed2149-8300-4c65-bc47-f6a0655d5574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:52.852673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed2149-8300-4c65-bc47-f6a0655d5574 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:52.852995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed2149-8300-4c65-bc47-f6a0655d5574 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:52.853262 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed2149-8300-4c65-bc47-f6a0655d5574 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:52.864133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed2149-8300-4c65-bc47-f6a0655d5574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-d5ed2149-8300-4c65-bc47-f6a0655d5574 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:52.869670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:52.870586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:52.871519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 135/276] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:29:52.876579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed2149-8300-4c65-bc47-f6a0655d5574 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:52.908407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:52.940956 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:52.959256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03540c6c-59bc-4697-a941-9a4568053ce1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:52.990803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed2149-8300-4c65-bc47-f6a0655d5574 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:52.999274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bda8bd-7813-40c3-a341-7860ffca73b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:52.999907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bda8bd-7813-40c3-a341-7860ffca73b5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:53.000181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bda8bd-7813-40c3-a341-7860ffca73b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:53.000450 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bda8bd-7813-40c3-a341-7860ffca73b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:53.002045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bda8bd-7813-40c3-a341-7860ffca73b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:53.003702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bda8bd-7813-40c3-a341-7860ffca73b5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:53.005794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 142/277] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:53.013250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bda8bd-7813-40c3-a341-7860ffca73b5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:53.039498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bf296-9977-49b5-848b-2dea0cfb2168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:53.040095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bf296-9977-49b5-848b-2dea0cfb2168 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:53.040380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bf296-9977-49b5-848b-2dea0cfb2168 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:53.040654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bf296-9977-49b5-848b-2dea0cfb2168 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:53.041325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bf296-9977-49b5-848b-2dea0cfb2168 None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:53.041635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bf296-9977-49b5-848b-2dea0cfb2168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:53.043053 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bf296-9977-49b5-848b-2dea0cfb2168 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:53.051777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 136/278] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:53.051777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bf296-9977-49b5-848b-2dea0cfb2168 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:53.129766 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bda8bd-7813-40c3-a341-7860ffca73b5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:53.156012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:53.169121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bf296-9977-49b5-848b-2dea0cfb2168 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:53.172254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a4f9a-8e8b-4d72-9a1e-7cd0dff2639a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:53.172871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a4f9a-8e8b-4d72-9a1e-7cd0dff2639a None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:53.173156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a4f9a-8e8b-4d72-9a1e-7cd0dff2639a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:53.173485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a4f9a-8e8b-4d72-9a1e-7cd0dff2639a None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:53.174151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a4f9a-8e8b-4d72-9a1e-7cd0dff2639a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:53.175649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a4f9a-8e8b-4d72-9a1e-7cd0dff2639a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:53.213299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4d126-63eb-462d-890d-cbf37eaa96a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:53.213299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4d126-63eb-462d-890d-cbf37eaa96a9 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:53.213299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4d126-63eb-462d-890d-cbf37eaa96a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:53.213299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4d126-63eb-462d-890d-cbf37eaa96a9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:53.216886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4d126-63eb-462d-890d-cbf37eaa96a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-dfa4d126-63eb-462d-890d-cbf37eaa96a9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:53.222911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:53.223780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:53.224587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 137/279] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:53.225267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4d126-63eb-462d-890d-cbf37eaa96a9 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:53.297488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 143/280] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:29:53 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:53.298141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a4f9a-8e8b-4d72-9a1e-7cd0dff2639a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:53.319831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:53.366129 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4d126-63eb-462d-890d-cbf37eaa96a9 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:53.415295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cac51-980e-42d8-83bc-06fb9e4ab703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:53.415951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cac51-980e-42d8-83bc-06fb9e4ab703 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:53.416232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cac51-980e-42d8-83bc-06fb9e4ab703 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:53.416505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cac51-980e-42d8-83bc-06fb9e4ab703 None admin] PATH_INFO: `/v3/users/4e14f53e721e49f1be4462e0b6e701e9` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:53.435052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a4f9a-8e8b-4d72-9a1e-7cd0dff2639a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:53.436464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cac51-980e-42d8-83bc-06fb9e4ab703 None admin] RBAC: Authorizing `identity:get_user(user_id=4e14f53e721e49f1be4462e0b6e701e9)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:53.438180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cac51-980e-42d8-83bc-06fb9e4ab703 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:53.440858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 138/281] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:53 2022] GET /identity/v3/users/4e14f53e721e49f1be4462e0b6e701e9 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:53.449048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cac51-980e-42d8-83bc-06fb9e4ab703 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:53.461983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:53.485417 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49473d79-d1bf-44fb-bad1-6efd38afb3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:53.485942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49473d79-d1bf-44fb-bad1-6efd38afb3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:53.486224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49473d79-d1bf-44fb-bad1-6efd38afb3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:53.486556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49473d79-d1bf-44fb-bad1-6efd38afb3b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:53.487237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49473d79-d1bf-44fb-bad1-6efd38afb3b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:53.487543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49473d79-d1bf-44fb-bad1-6efd38afb3b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:53.489063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49473d79-d1bf-44fb-bad1-6efd38afb3b8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:53.501334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 144/282] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:53.509156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49473d79-d1bf-44fb-bad1-6efd38afb3b8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:53.585715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cac51-980e-42d8-83bc-06fb9e4ab703 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:53.615654 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:53.634983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8524c-a91e-40ce-969f-6a3a2646da0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:53.635606 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8524c-a91e-40ce-969f-6a3a2646da0a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:53.635895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8524c-a91e-40ce-969f-6a3a2646da0a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:53.636170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8524c-a91e-40ce-969f-6a3a2646da0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:53.636914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8524c-a91e-40ce-969f-6a3a2646da0a None admin] RBAC: Adding query filter params (role.id=4d98a2b16f8f42bca3e5dac0374cdb54, user.id=4e14f53e721e49f1be4462e0b6e701e9, scope.system=all) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:53.637233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8524c-a91e-40ce-969f-6a3a2646da0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:53.638706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8524c-a91e-40ce-969f-6a3a2646da0a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:53.645803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 139/283] 10.222.0.58 () {56 vars in 1308 bytes} [Mon Aug 29 23:29:53 2022] GET /identity/v3/role_assignments?user.id=4e14f53e721e49f1be4462e0b6e701e9&scope.system=all&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:53.663400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49473d79-d1bf-44fb-bad1-6efd38afb3b8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:53.703800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f0642-f271-40d4-a4b5-2292a024148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:53.704459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f0642-f271-40d4-a4b5-2292a024148e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:53.704804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f0642-f271-40d4-a4b5-2292a024148e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:53.705084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f0642-f271-40d4-a4b5-2292a024148e None admin] PATH_INFO: `/v3/users/d55e95742e7c43e38a9d970fddcd335b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:53.706394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f0642-f271-40d4-a4b5-2292a024148e None admin] RBAC: Authorizing `identity:get_user(user_id=d55e95742e7c43e38a9d970fddcd335b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:53.708005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f0642-f271-40d4-a4b5-2292a024148e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:53.709789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 145/284] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:53 2022] GET /identity/v3/users/d55e95742e7c43e38a9d970fddcd335b => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:53.716688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8524c-a91e-40ce-969f-6a3a2646da0a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:53.799581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:53.834177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8524c-a91e-40ce-969f-6a3a2646da0a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:53.873475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea0db9-64fe-41c1-89cd-a70ee2a9b30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:53.874118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0db9-64fe-41c1-89cd-a70ee2a9b30b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:53.874401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0db9-64fe-41c1-89cd-a70ee2a9b30b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:53.874702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0db9-64fe-41c1-89cd-a70ee2a9b30b None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:53.875845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea0db9-64fe-41c1-89cd-a70ee2a9b30b None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:53.877420 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea0db9-64fe-41c1-89cd-a70ee2a9b30b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:53.879380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 140/285] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:53 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:29:53.887583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f0642-f271-40d4-a4b5-2292a024148e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:53.998413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f0642-f271-40d4-a4b5-2292a024148e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:54.036460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae94914-fa49-4c90-86e2-97b51d6e1764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:54.037427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae94914-fa49-4c90-86e2-97b51d6e1764 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:54.037882 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae94914-fa49-4c90-86e2-97b51d6e1764 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:54.038399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae94914-fa49-4c90-86e2-97b51d6e1764 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/users/d55e95742e7c43e38a9d970fddcd335b/roles/6f434b187a404b7cb1e9bb8303ce59b5` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:54.041283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae94914-fa49-4c90-86e2-97b51d6e1764 None admin] RBAC: Authorizing `identity:create_grant(project_id=45aa2f6db17f4df1b7f892a95d31b8fc, user_id=d55e95742e7c43e38a9d970fddcd335b, role_id=6f434b187a404b7cb1e9bb8303ce59b5)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:54.042990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae94914-fa49-4c90-86e2-97b51d6e1764 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:54.047861 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e42b1a-81d0-477f-a090-7ab5ad1b98e6 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:54.048326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e42b1a-81d0-477f-a090-7ab5ad1b98e6 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:54.048544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e42b1a-81d0-477f-a090-7ab5ad1b98e6 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:54.049998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 141/286] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:29:54.055485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24014-73d5-49a1-9a5f-06745c9a94d9 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:54.055826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24014-73d5-49a1-9a5f-06745c9a94d9 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:54.056103 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24014-73d5-49a1-9a5f-06745c9a94d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:54.075729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-3fd24014-73d5-49a1-9a5f-06745c9a94d9 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:54.076999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3fd24014-73d5-49a1-9a5f-06745c9a94d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:54.130463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 146/287] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:29:53 2022] PUT /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/users/d55e95742e7c43e38a9d970fddcd335b/roles/6f434b187a404b7cb1e9bb8303ce59b5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:29:54.169270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 142/288] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:29:54.452267 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae94914-fa49-4c90-86e2-97b51d6e1764 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:54.455494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:54.586657 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae94914-fa49-4c90-86e2-97b51d6e1764 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:54.629889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cad3d8-8e90-4cb2-99ff-dcf11589ed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:54.630471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cad3d8-8e90-4cb2-99ff-dcf11589ed22 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:54.630729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cad3d8-8e90-4cb2-99ff-dcf11589ed22 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:54.631003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cad3d8-8e90-4cb2-99ff-dcf11589ed22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:54.651399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cad3d8-8e90-4cb2-99ff-dcf11589ed22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-78cad3d8-8e90-4cb2-99ff-dcf11589ed22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:54.656824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:54.657692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:54.658930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 147/289] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:54.669822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24014-73d5-49a1-9a5f-06745c9a94d9 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:54.756994 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:54.793686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24014-73d5-49a1-9a5f-06745c9a94d9 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:54.838288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ee895-64ac-4ff2-9454-568bf20e7e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:54.838939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ee895-64ac-4ff2-9454-568bf20e7e4c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:54.839213 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ee895-64ac-4ff2-9454-568bf20e7e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:54.839482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ee895-64ac-4ff2-9454-568bf20e7e4c None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:54.840160 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ee895-64ac-4ff2-9454-568bf20e7e4c None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:54.840459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ee895-64ac-4ff2-9454-568bf20e7e4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:54.841843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ee895-64ac-4ff2-9454-568bf20e7e4c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:54.851871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 143/290] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:54.857752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cad3d8-8e90-4cb2-99ff-dcf11589ed22 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:54.954857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:55.005014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cad3d8-8e90-4cb2-99ff-dcf11589ed22 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:55.052222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efce72-b393-4b2c-8a3e-a4b46ecc87da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:55.052866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efce72-b393-4b2c-8a3e-a4b46ecc87da None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:55.053147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efce72-b393-4b2c-8a3e-a4b46ecc87da None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:55.053423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efce72-b393-4b2c-8a3e-a4b46ecc87da None admin] PATH_INFO: `/v3/users/5665c4e3e54d4031b144eea4d354c2a3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:55.054660 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efce72-b393-4b2c-8a3e-a4b46ecc87da None admin] RBAC: Authorizing `identity:get_user(user_id=5665c4e3e54d4031b144eea4d354c2a3)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:55.056115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efce72-b393-4b2c-8a3e-a4b46ecc87da None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:55.057694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 148/291] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:54 2022] GET /identity/v3/users/5665c4e3e54d4031b144eea4d354c2a3 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:55.063615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ee895-64ac-4ff2-9454-568bf20e7e4c None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:55.246910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ee895-64ac-4ff2-9454-568bf20e7e4c None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:55.276161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:55.281854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87eb8bc-16a0-497c-b1cb-f453ca2b5336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:55.282497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eb8bc-16a0-497c-b1cb-f453ca2b5336 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:55.282905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eb8bc-16a0-497c-b1cb-f453ca2b5336 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:55.283175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eb8bc-16a0-497c-b1cb-f453ca2b5336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:55.365229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87eb8bc-16a0-497c-b1cb-f453ca2b5336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:55.367329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87eb8bc-16a0-497c-b1cb-f453ca2b5336 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:55.470623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 144/292] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:55.476415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efce72-b393-4b2c-8a3e-a4b46ecc87da None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:55.601758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efce72-b393-4b2c-8a3e-a4b46ecc87da None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:55.631653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:55.644446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401ba41-1f3e-4f6b-810a-b77c2222ec87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:55.645079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401ba41-1f3e-4f6b-810a-b77c2222ec87 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:55.645366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401ba41-1f3e-4f6b-810a-b77c2222ec87 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:55.645644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401ba41-1f3e-4f6b-810a-b77c2222ec87 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:55.647087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401ba41-1f3e-4f6b-810a-b77c2222ec87 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:55.648698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401ba41-1f3e-4f6b-810a-b77c2222ec87 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:55.650699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 149/293] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:55 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:55.657903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87eb8bc-16a0-497c-b1cb-f453ca2b5336 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:55.703517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:55.797936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87eb8bc-16a0-497c-b1cb-f453ca2b5336 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:55.849252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bdbf40-0133-43e3-82a7-aa97e27999ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:55.849867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdbf40-0133-43e3-82a7-aa97e27999ec None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:55.850159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdbf40-0133-43e3-82a7-aa97e27999ec None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:55.850522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdbf40-0133-43e3-82a7-aa97e27999ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:55.851862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bdbf40-0133-43e3-82a7-aa97e27999ec None admin] RBAC: Adding query filter params (role.id=4d98a2b16f8f42bca3e5dac0374cdb54, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc, user.id=5665c4e3e54d4031b144eea4d354c2a3) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:55.852120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bdbf40-0133-43e3-82a7-aa97e27999ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:55.853559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bdbf40-0133-43e3-82a7-aa97e27999ec None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:55.863041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 145/294] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:55 2022] GET /identity/v3/role_assignments?user.id=5665c4e3e54d4031b144eea4d354c2a3&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:55.926213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e602cc-7b3d-4c65-b80a-b76154fa00ef None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:55.926540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e602cc-7b3d-4c65-b80a-b76154fa00ef None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:55.926813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e602cc-7b3d-4c65-b80a-b76154fa00ef None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:55.928460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 150/295] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:55.934410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f068-ebff-49b7-8768-9fc5c552a068 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:55.934729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f068-ebff-49b7-8768-9fc5c552a068 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:55.935007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f068-ebff-49b7-8768-9fc5c552a068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:55.967654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-9350f068-ebff-49b7-8768-9fc5c552a068 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:55.968986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9350f068-ebff-49b7-8768-9fc5c552a068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:56.062045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 146/296] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:56.186846 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf0b2c-b3fd-4628-812d-d6aa57b7996b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:56.187326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf0b2c-b3fd-4628-812d-d6aa57b7996b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:56.187326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf0b2c-b3fd-4628-812d-d6aa57b7996b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:56.188332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8df29-c206-4e59-a34c-49da22c67d28 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:56.188710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 151/297] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:56.189223 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8df29-c206-4e59-a34c-49da22c67d28 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:56.189730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8df29-c206-4e59-a34c-49da22c67d28 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:56.191312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 147/298] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:29:56.196951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c4c2-2987-40a0-9a56-97e837605012 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:56.197236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c4c2-2987-40a0-9a56-97e837605012 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:56.197511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c4c2-2987-40a0-9a56-97e837605012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:56.197862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a530da6-9c56-44f5-87f9-7426021bceb7 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:56.198331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a530da6-9c56-44f5-87f9-7426021bceb7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:56.198839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a530da6-9c56-44f5-87f9-7426021bceb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:56.220516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-34d4c4c2-2987-40a0-9a56-97e837605012 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:56.221682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-34d4c4c2-2987-40a0-9a56-97e837605012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:56.227714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-0a530da6-9c56-44f5-87f9-7426021bceb7 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:56.229226 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-0a530da6-9c56-44f5-87f9-7426021bceb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:56.250694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:56.305832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 152/299] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:56.323508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 148/300] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:29:56.370255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4c4c2-2987-40a0-9a56-97e837605012 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:56.488267 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4c4c2-2987-40a0-9a56-97e837605012 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:56.543202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:56.551219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dcc16-2412-40a5-af3d-f8be23aef49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:56.551920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dcc16-2412-40a5-af3d-f8be23aef49d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:56.552242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dcc16-2412-40a5-af3d-f8be23aef49d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:56.552521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dcc16-2412-40a5-af3d-f8be23aef49d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:56.563518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419dcc16-2412-40a5-af3d-f8be23aef49d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-419dcc16-2412-40a5-af3d-f8be23aef49d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:56.569010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:56.569923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:29:56.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 153/301] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:56.576281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a530da6-9c56-44f5-87f9-7426021bceb7 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:56.626919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dcc16-2412-40a5-af3d-f8be23aef49d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:56.690351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a530da6-9c56-44f5-87f9-7426021bceb7 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:56.739136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02561af-5a27-4c59-9b2f-b29337928bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:56.739663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02561af-5a27-4c59-9b2f-b29337928bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:56.739940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02561af-5a27-4c59-9b2f-b29337928bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:56.740208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02561af-5a27-4c59-9b2f-b29337928bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:56.740986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02561af-5a27-4c59-9b2f-b29337928bc8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:56.741404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02561af-5a27-4c59-9b2f-b29337928bc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:56.743142 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02561af-5a27-4c59-9b2f-b29337928bc8 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:56.745815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dcc16-2412-40a5-af3d-f8be23aef49d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:56.753213 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 149/302] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:56.755021 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02561af-5a27-4c59-9b2f-b29337928bc8 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:56.774580 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:56.782952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bdad1-6e33-4625-bbb7-268b139ab0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:56.783546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bdad1-6e33-4625-bbb7-268b139ab0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:56.783824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bdad1-6e33-4625-bbb7-268b139ab0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:56.784094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bdad1-6e33-4625-bbb7-268b139ab0b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:56.785907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bdad1-6e33-4625-bbb7-268b139ab0b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:56.786880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bdad1-6e33-4625-bbb7-268b139ab0b4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:56.789250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 154/303] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:56.794177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a0098-87fc-4d01-9f73-40db36c22cc7 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:56.794177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a0098-87fc-4d01-9f73-40db36c22cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:56.794177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a0098-87fc-4d01-9f73-40db36c22cc7 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:56.794177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 155/304] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:56.794177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a0098-87fc-4d01-9f73-40db36c22cc7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:56.849278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:56.860078 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02561af-5a27-4c59-9b2f-b29337928bc8 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:56.894347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad53302-358e-4a57-8286-663db17c723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:56.895066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad53302-358e-4a57-8286-663db17c723a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:56.895296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a0098-87fc-4d01-9f73-40db36c22cc7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:56.895413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad53302-358e-4a57-8286-663db17c723a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:56.895597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad53302-358e-4a57-8286-663db17c723a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:56.906396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad53302-358e-4a57-8286-663db17c723a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-0ad53302-358e-4a57-8286-663db17c723a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:56.911649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:56.912719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 150/305] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:56.915465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad53302-358e-4a57-8286-663db17c723a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:56.934761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be901fc-886f-417c-a721-bee383e4af2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:56.935414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be901fc-886f-417c-a721-bee383e4af2c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:56.935685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be901fc-886f-417c-a721-bee383e4af2c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:56.935947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be901fc-886f-417c-a721-bee383e4af2c None admin] PATH_INFO: `/v3/users/c7ae99a1cf124aa781eda159f3c6c470` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:56.950282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be901fc-886f-417c-a721-bee383e4af2c None admin] RBAC: Authorizing `identity:get_user(user_id=c7ae99a1cf124aa781eda159f3c6c470)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:56.951774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be901fc-886f-417c-a721-bee383e4af2c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:56.953370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 156/306] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:56 2022] GET /identity/v3/users/c7ae99a1cf124aa781eda159f3c6c470 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:56.956038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdc55e-8271-4306-904c-75d549c51008 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:56.956325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdc55e-8271-4306-904c-75d549c51008 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:56.956966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdc55e-8271-4306-904c-75d549c51008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:56.974614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-22bdc55e-8271-4306-904c-75d549c51008 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:56.975866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-22bdc55e-8271-4306-904c-75d549c51008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:56.990977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:57.015576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad53302-358e-4a57-8286-663db17c723a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:57.043707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 157/307] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:57.044573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdc55e-8271-4306-904c-75d549c51008 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:57.052297 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df73448-9719-4917-b9ef-422d299b4f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:57.052902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df73448-9719-4917-b9ef-422d299b4f5a None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:57.053185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df73448-9719-4917-b9ef-422d299b4f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:57.053449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df73448-9719-4917-b9ef-422d299b4f5a None admin] PATH_INFO: `/v3/groups` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:57.054646 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df73448-9719-4917-b9ef-422d299b4f5a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:57.056145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df73448-9719-4917-b9ef-422d299b4f5a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:57.108857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:57.138457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 151/308] 10.222.0.58 () {60 vars in 1134 bytes} [Mon Aug 29 23:29:56 2022] POST /identity/v3/groups => generated 229 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:29:57.138457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df73448-9719-4917-b9ef-422d299b4f5a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:57.183941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdc55e-8271-4306-904c-75d549c51008 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:57.227310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dece079-dae6-4676-8ec1-47714504ca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:57.228230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dece079-dae6-4676-8ec1-47714504ca1b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:57.229083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dece079-dae6-4676-8ec1-47714504ca1b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:57.229375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dece079-dae6-4676-8ec1-47714504ca1b None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:57.230348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dece079-dae6-4676-8ec1-47714504ca1b None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:57.230707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dece079-dae6-4676-8ec1-47714504ca1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:57.232441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dece079-dae6-4676-8ec1-47714504ca1b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:57.245978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 158/309] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:57.252042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dece079-dae6-4676-8ec1-47714504ca1b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:57.254800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df73448-9719-4917-b9ef-422d299b4f5a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:57.301451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdbefd-897d-4aa5-9583-7ca06ec06898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:57.302148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbefd-897d-4aa5-9583-7ca06ec06898 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:57.302453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbefd-897d-4aa5-9583-7ca06ec06898 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:57.302807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbefd-897d-4aa5-9583-7ca06ec06898 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:57.303548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdbefd-897d-4aa5-9583-7ca06ec06898 None admin] RBAC: Adding query filter params (role.id=6f434b187a404b7cb1e9bb8303ce59b5, user.id=c7ae99a1cf124aa781eda159f3c6c470, scope.system=all) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:57.303894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdbefd-897d-4aa5-9583-7ca06ec06898 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:57.305398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdbefd-897d-4aa5-9583-7ca06ec06898 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:57.309780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:57.313064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 152/310] 10.222.0.58 () {56 vars in 1308 bytes} [Mon Aug 29 23:29:57 2022] GET /identity/v3/role_assignments?user.id=c7ae99a1cf124aa781eda159f3c6c470&scope.system=all&role.id=6f434b187a404b7cb1e9bb8303ce59b5 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:57.324288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdbefd-897d-4aa5-9583-7ca06ec06898 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:57.390509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dece079-dae6-4676-8ec1-47714504ca1b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:57.402850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:57.431304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdc6ea-51a1-412d-98ce-0d3ae8fb4df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:57.432089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdc6ea-51a1-412d-98ce-0d3ae8fb4df6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:57.432382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdc6ea-51a1-412d-98ce-0d3ae8fb4df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:57.432640 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdc6ea-51a1-412d-98ce-0d3ae8fb4df6 None admin] PATH_INFO: `/v3/users/4e14f53e721e49f1be4462e0b6e701e9` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:57.433848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdc6ea-51a1-412d-98ce-0d3ae8fb4df6 None admin] RBAC: Authorizing `identity:get_user(user_id=4e14f53e721e49f1be4462e0b6e701e9)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:57.435617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdc6ea-51a1-412d-98ce-0d3ae8fb4df6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:57.438564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 159/311] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:57 2022] GET /identity/v3/users/4e14f53e721e49f1be4462e0b6e701e9 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:57.444276 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdc6ea-51a1-412d-98ce-0d3ae8fb4df6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:57.472958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdbefd-897d-4aa5-9583-7ca06ec06898 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:57.583152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdc6ea-51a1-412d-98ce-0d3ae8fb4df6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:57.584423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e30af-1694-4355-ac6e-6408cfa8df58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:57.585638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e30af-1694-4355-ac6e-6408cfa8df58 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:57.585997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e30af-1694-4355-ac6e-6408cfa8df58 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:57.586330 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e30af-1694-4355-ac6e-6408cfa8df58 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:57.597391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e30af-1694-4355-ac6e-6408cfa8df58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-e18e30af-1694-4355-ac6e-6408cfa8df58 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:57.603993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:57.605152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:57.605933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 153/312] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:29:57.612668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e30af-1694-4355-ac6e-6408cfa8df58 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:57.627088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60a978-e8cb-47d7-b507-6c3d4f4c4f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:57.628185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60a978-e8cb-47d7-b507-6c3d4f4c4f04 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:57.628665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60a978-e8cb-47d7-b507-6c3d4f4c4f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:57.629030 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60a978-e8cb-47d7-b507-6c3d4f4c4f04 None admin] PATH_INFO: `/v3/system/users/4e14f53e721e49f1be4462e0b6e701e9/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:57.630716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60a978-e8cb-47d7-b507-6c3d4f4c4f04 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4e14f53e721e49f1be4462e0b6e701e9, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:57.632557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60a978-e8cb-47d7-b507-6c3d4f4c4f04 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:57.681477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:57.732203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e30af-1694-4355-ac6e-6408cfa8df58 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:57.735280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 160/313] 10.222.0.58 () {58 vars in 1334 bytes} [Mon Aug 29 23:29:57 2022] PUT /identity/v3/system/users/4e14f53e721e49f1be4462e0b6e701e9/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:29:57.779667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886049f6-5015-436c-bb58-d1ce7bca55b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:57.780554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886049f6-5015-436c-bb58-d1ce7bca55b7 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:57.781035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886049f6-5015-436c-bb58-d1ce7bca55b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:57.781464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886049f6-5015-436c-bb58-d1ce7bca55b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:57.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886049f6-5015-436c-bb58-d1ce7bca55b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:57.782884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886049f6-5015-436c-bb58-d1ce7bca55b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:57.784491 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886049f6-5015-436c-bb58-d1ce7bca55b7 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:57.795478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 154/314] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:29:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:57.801383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60a978-e8cb-47d7-b507-6c3d4f4c4f04 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:57.854869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:57.921572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60a978-e8cb-47d7-b507-6c3d4f4c4f04 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:57.957901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02319-597a-41c6-8e76-beda58200144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:57.958561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02319-597a-41c6-8e76-beda58200144 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:57.958840 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02319-597a-41c6-8e76-beda58200144 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:57.959106 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02319-597a-41c6-8e76-beda58200144 None admin] PATH_INFO: `/v3/users/d55e95742e7c43e38a9d970fddcd335b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:57.960365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe02319-597a-41c6-8e76-beda58200144 None admin] RBAC: Authorizing `identity:get_user(user_id=d55e95742e7c43e38a9d970fddcd335b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:57.962204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe02319-597a-41c6-8e76-beda58200144 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:57.964050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 161/315] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:29:57 2022] GET /identity/v3/users/d55e95742e7c43e38a9d970fddcd335b => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:57.969536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886049f6-5015-436c-bb58-d1ce7bca55b7 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:58.076822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886049f6-5015-436c-bb58-d1ce7bca55b7 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:58.101048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:58.114341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffce72-7773-4845-9ea1-d47ab2a850d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:58.114341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffce72-7773-4845-9ea1-d47ab2a850d0 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:58.114341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffce72-7773-4845-9ea1-d47ab2a850d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:58.114341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffce72-7773-4845-9ea1-d47ab2a850d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:58.186569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffce72-7773-4845-9ea1-d47ab2a850d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:58.188022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffce72-7773-4845-9ea1-d47ab2a850d0 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:58.298785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 155/316] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:29:58.300306 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02319-597a-41c6-8e76-beda58200144 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:58.422737 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02319-597a-41c6-8e76-beda58200144 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:58.430952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:58.458795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839efd41-aa75-4e83-8d8b-1e7015b13f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:58.459433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839efd41-aa75-4e83-8d8b-1e7015b13f16 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:58.459715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839efd41-aa75-4e83-8d8b-1e7015b13f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:58.460540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839efd41-aa75-4e83-8d8b-1e7015b13f16 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:58.461911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839efd41-aa75-4e83-8d8b-1e7015b13f16 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:58.463390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839efd41-aa75-4e83-8d8b-1e7015b13f16 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:58.465697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 162/317] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:29:58 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:58.470953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffce72-7773-4845-9ea1-d47ab2a850d0 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:58.510393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:58.579444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffce72-7773-4845-9ea1-d47ab2a850d0 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:58.614406 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffe35b-1faa-45eb-9211-47f2b020d6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:58.615035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffe35b-1faa-45eb-9211-47f2b020d6de None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:58.615617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffe35b-1faa-45eb-9211-47f2b020d6de None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:58.615889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffe35b-1faa-45eb-9211-47f2b020d6de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:58.616938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffe35b-1faa-45eb-9211-47f2b020d6de None admin] RBAC: Adding query filter params (role.id=6f434b187a404b7cb1e9bb8303ce59b5, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc, user.id=d55e95742e7c43e38a9d970fddcd335b) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:29:58.617250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffe35b-1faa-45eb-9211-47f2b020d6de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:58.618767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffe35b-1faa-45eb-9211-47f2b020d6de None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:58.625211 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 156/318] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:29:58 2022] GET /identity/v3/role_assignments?user.id=d55e95742e7c43e38a9d970fddcd335b&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=6f434b187a404b7cb1e9bb8303ce59b5 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:59.222166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc10991-ae11-4c2b-a676-66416bbd0090 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:59.222477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc10991-ae11-4c2b-a676-66416bbd0090 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:59.222761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc10991-ae11-4c2b-a676-66416bbd0090 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:59.224334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 163/319] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:59.230572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b04dd-0225-42e1-9459-919967a516d6 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:59.230881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b04dd-0225-42e1-9459-919967a516d6 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:59.231143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b04dd-0225-42e1-9459-919967a516d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:59.250067 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-070b04dd-0225-42e1-9459-919967a516d6 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:59.251344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-070b04dd-0225-42e1-9459-919967a516d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:59.334974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 157/320] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:59.358868 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4a63a-0632-4ccc-840a-2a40e86f6c66 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:59.359429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4a63a-0632-4ccc-840a-2a40e86f6c66 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:59.359429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4a63a-0632-4ccc-840a-2a40e86f6c66 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:59.360953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 164/321] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:59.367294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48232d-1236-4108-91b0-abbde89140ca None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:59.367641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48232d-1236-4108-91b0-abbde89140ca None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:59.367853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48232d-1236-4108-91b0-abbde89140ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:59.387520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-db48232d-1236-4108-91b0-abbde89140ca None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:59.388749 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-db48232d-1236-4108-91b0-abbde89140ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:29:59.402383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:59.472284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 158/322] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:29:59.613555 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4a63a-0632-4ccc-840a-2a40e86f6c66 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:59.729459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4a63a-0632-4ccc-840a-2a40e86f6c66 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:59.757561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48232d-1236-4108-91b0-abbde89140ca None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:59.764612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88423f-2835-40cb-8db7-84c11bce6f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:59.765443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88423f-2835-40cb-8db7-84c11bce6f04 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:59.765559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88423f-2835-40cb-8db7-84c11bce6f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:59.765827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88423f-2835-40cb-8db7-84c11bce6f04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:59.766949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88423f-2835-40cb-8db7-84c11bce6f04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:59.768399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88423f-2835-40cb-8db7-84c11bce6f04 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:59.769944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 165/323] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:29:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:29:59.771079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d63ce5-5811-49e5-b45f-9937ae253359 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:59.771618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d63ce5-5811-49e5-b45f-9937ae253359 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:59.771886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d63ce5-5811-49e5-b45f-9937ae253359 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:59.773252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 166/324] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:29:59.777696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d63ce5-5811-49e5-b45f-9937ae253359 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:29:59.823253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:59.861111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48232d-1236-4108-91b0-abbde89140ca None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:59.880490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d63ce5-5811-49e5-b45f-9937ae253359 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:29:59.899011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db2e54-8a59-4fe7-9636-da1448fddb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:59.899647 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db2e54-8a59-4fe7-9636-da1448fddb2c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:59.899928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db2e54-8a59-4fe7-9636-da1448fddb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:59.900197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db2e54-8a59-4fe7-9636-da1448fddb2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:59.910658 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db2e54-8a59-4fe7-9636-da1448fddb2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:29:59.915748 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d2ce0-6a12-46f6-86fd-a6ff662fd9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-40db2e54-8a59-4fe7-9636-da1448fddb2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:29:59.915885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:29:59.916756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:29:59.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:29:59.918143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d2ce0-6a12-46f6-86fd-a6ff662fd9dc None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:59.918143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d2ce0-6a12-46f6-86fd-a6ff662fd9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:59.918143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d2ce0-6a12-46f6-86fd-a6ff662fd9dc None admin] PATH_INFO: `/v3/groups` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:59.918318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 159/325] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:29:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:29:59.918390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d2ce0-6a12-46f6-86fd-a6ff662fd9dc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:29:59.919022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d2ce0-6a12-46f6-86fd-a6ff662fd9dc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:29:59.919732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2203ce-14c4-4fa3-ab9d-f6baf483381b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:29:59.920018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2203ce-14c4-4fa3-ab9d-f6baf483381b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:29:59.920292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2203ce-14c4-4fa3-ab9d-f6baf483381b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:29:59.934240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:29:59.940031 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-1d2203ce-14c4-4fa3-ab9d-f6baf483381b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:29:59.941248 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1d2203ce-14c4-4fa3-ab9d-f6baf483381b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:00.006317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 160/326] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:00.007802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2203ce-14c4-4fa3-ab9d-f6baf483381b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:00.025907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 167/327] 10.222.0.58 () {60 vars in 1134 bytes} [Mon Aug 29 23:29:59 2022] POST /identity/v3/groups => generated 226 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:30:00.131020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2203ce-14c4-4fa3-ab9d-f6baf483381b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:00.168854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269098a8-0235-4792-bb2c-a55983b08836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:00.169471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269098a8-0235-4792-bb2c-a55983b08836 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:00.169805 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269098a8-0235-4792-bb2c-a55983b08836 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:00.170080 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269098a8-0235-4792-bb2c-a55983b08836 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:00.170789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269098a8-0235-4792-bb2c-a55983b08836 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:00.171102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269098a8-0235-4792-bb2c-a55983b08836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:00.172477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269098a8-0235-4792-bb2c-a55983b08836 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:00.175121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:00.183719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 161/328] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:30:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:00.189098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d2ce0-6a12-46f6-86fd-a6ff662fd9dc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:00.300935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269098a8-0235-4792-bb2c-a55983b08836 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:00.317509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d2ce0-6a12-46f6-86fd-a6ff662fd9dc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:00.320162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:00.366804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae1f92-c66e-4e12-b235-ea3fa4b98c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:00.367498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae1f92-c66e-4e12-b235-ea3fa4b98c4f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:00.367764 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae1f92-c66e-4e12-b235-ea3fa4b98c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:00.368039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae1f92-c66e-4e12-b235-ea3fa4b98c4f None admin] PATH_INFO: `/v3/users/c7ae99a1cf124aa781eda159f3c6c470` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:00.369305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae1f92-c66e-4e12-b235-ea3fa4b98c4f None admin] RBAC: Authorizing `identity:get_user(user_id=c7ae99a1cf124aa781eda159f3c6c470)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:00.370806 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae1f92-c66e-4e12-b235-ea3fa4b98c4f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:00.372743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 168/329] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:30:00 2022] GET /identity/v3/users/c7ae99a1cf124aa781eda159f3c6c470 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:00.378696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae1f92-c66e-4e12-b235-ea3fa4b98c4f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:00.433721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269098a8-0235-4792-bb2c-a55983b08836 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:00.486107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d8284-8cc2-4832-b200-9e89a133acd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:00.486925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d8284-8cc2-4832-b200-9e89a133acd6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:00.487433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d8284-8cc2-4832-b200-9e89a133acd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:00.487712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d8284-8cc2-4832-b200-9e89a133acd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:00.508866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d8284-8cc2-4832-b200-9e89a133acd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-dc5d8284-8cc2-4832-b200-9e89a133acd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:00.515456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:00.516719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:00.517557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:00.518351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 162/330] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:00.525288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d8284-8cc2-4832-b200-9e89a133acd6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:00.530041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae1f92-c66e-4e12-b235-ea3fa4b98c4f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:00.577340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98017d6-3999-4b58-ab70-fda12c33e659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:00.577929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98017d6-3999-4b58-ab70-fda12c33e659 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:00.578206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98017d6-3999-4b58-ab70-fda12c33e659 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:00.578510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98017d6-3999-4b58-ab70-fda12c33e659 None admin] PATH_INFO: `/v3/system/users/c7ae99a1cf124aa781eda159f3c6c470/roles/6f434b187a404b7cb1e9bb8303ce59b5` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:00.579964 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98017d6-3999-4b58-ab70-fda12c33e659 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c7ae99a1cf124aa781eda159f3c6c470, role_id=6f434b187a404b7cb1e9bb8303ce59b5)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:00.581387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98017d6-3999-4b58-ab70-fda12c33e659 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:00.604328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:00.652755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d8284-8cc2-4832-b200-9e89a133acd6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:00.661662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 169/331] 10.222.0.58 () {58 vars in 1334 bytes} [Mon Aug 29 23:30:00 2022] PUT /identity/v3/system/users/c7ae99a1cf124aa781eda159f3c6c470/roles/6f434b187a404b7cb1e9bb8303ce59b5 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:30:00.700174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269c26f-5a55-4142-ba79-0a52b627eabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:00.703139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269c26f-5a55-4142-ba79-0a52b627eabd None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:00.703139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269c26f-5a55-4142-ba79-0a52b627eabd None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:00.703139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269c26f-5a55-4142-ba79-0a52b627eabd None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:00.703139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269c26f-5a55-4142-ba79-0a52b627eabd None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:00.703139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269c26f-5a55-4142-ba79-0a52b627eabd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:00.706447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269c26f-5a55-4142-ba79-0a52b627eabd None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:00.716976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 163/332] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:30:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:00.726456 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98017d6-3999-4b58-ab70-fda12c33e659 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:00.786614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:00.863770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98017d6-3999-4b58-ab70-fda12c33e659 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:00.905802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf89c79-8189-4aca-9365-9ebee3ff539a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:00.906450 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf89c79-8189-4aca-9365-9ebee3ff539a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:00.906734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf89c79-8189-4aca-9365-9ebee3ff539a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:00.907009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf89c79-8189-4aca-9365-9ebee3ff539a None admin] PATH_INFO: `/v3/users/4e14f53e721e49f1be4462e0b6e701e9` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:00.908521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf89c79-8189-4aca-9365-9ebee3ff539a None admin] RBAC: Authorizing `identity:get_user(user_id=4e14f53e721e49f1be4462e0b6e701e9)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:00.909972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf89c79-8189-4aca-9365-9ebee3ff539a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:00.911899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 170/333] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:30:00 2022] GET /identity/v3/users/4e14f53e721e49f1be4462e0b6e701e9 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:00.918280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269c26f-5a55-4142-ba79-0a52b627eabd None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:01.042558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269c26f-5a55-4142-ba79-0a52b627eabd None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:01.070814 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:01.092726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a0948-093f-435c-b53e-39d07707f2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:01.093356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0948-093f-435c-b53e-39d07707f2bf None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:01.101528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0948-093f-435c-b53e-39d07707f2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:01.101528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0948-093f-435c-b53e-39d07707f2bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:01.101528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a0948-093f-435c-b53e-39d07707f2bf None admin] RBAC: Adding query filter params (role.id=4d98a2b16f8f42bca3e5dac0374cdb54, user.id=4e14f53e721e49f1be4462e0b6e701e9, scope.system=all) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:01.101528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a0948-093f-435c-b53e-39d07707f2bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:01.101528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a0948-093f-435c-b53e-39d07707f2bf None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:01.116125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 164/334] 10.222.0.58 () {56 vars in 1308 bytes} [Mon Aug 29 23:30:00 2022] GET /identity/v3/role_assignments?user.id=4e14f53e721e49f1be4462e0b6e701e9&scope.system=all&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:02.278891 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9c857-62c5-4706-bf1c-74fc9f084c97 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:02.279399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9c857-62c5-4706-bf1c-74fc9f084c97 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:02.279474 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9c857-62c5-4706-bf1c-74fc9f084c97 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:02.281054 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 171/335] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:02.286387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518139bd-ae7a-41fb-ad67-d86d847dae2a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:02.286839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518139bd-ae7a-41fb-ad67-d86d847dae2a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:02.287019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518139bd-ae7a-41fb-ad67-d86d847dae2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:02.309740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-518139bd-ae7a-41fb-ad67-d86d847dae2a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:02.310941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-518139bd-ae7a-41fb-ad67-d86d847dae2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:02.395272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 165/336] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:02.555404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476edf-87d6-46ce-ad86-0a1ab03d3a01 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:02.555889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476edf-87d6-46ce-ad86-0a1ab03d3a01 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:02.556298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476edf-87d6-46ce-ad86-0a1ab03d3a01 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:02.558068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 172/337] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:02.563292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfa87e-03c2-4086-b3b9-50c33dc006c4 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:02.563826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfa87e-03c2-4086-b3b9-50c33dc006c4 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:02.564285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfa87e-03c2-4086-b3b9-50c33dc006c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:02.586637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-1fdfa87e-03c2-4086-b3b9-50c33dc006c4 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:02.588187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1fdfa87e-03c2-4086-b3b9-50c33dc006c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:02.684979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 166/338] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:02.974483 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07476edf-87d6-46ce-ad86-0a1ab03d3a01 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:03.103545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07476edf-87d6-46ce-ad86-0a1ab03d3a01 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:03.105857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:03.151206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ee14e-c794-4c8e-85fa-56a9931be507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:03.151909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ee14e-c794-4c8e-85fa-56a9931be507 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:03.152262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ee14e-c794-4c8e-85fa-56a9931be507 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:03.152624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ee14e-c794-4c8e-85fa-56a9931be507 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:03.171402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ee14e-c794-4c8e-85fa-56a9931be507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c73ee14e-c794-4c8e-85fa-56a9931be507 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:03.176390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:03.177326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:03.178808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 173/339] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:03.186078 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfa87e-03c2-4086-b3b9-50c33dc006c4 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:03.242509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:03.327913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ee14e-c794-4c8e-85fa-56a9931be507 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:03.330358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfa87e-03c2-4086-b3b9-50c33dc006c4 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:03.373067 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24189d0c-2f95-42c4-8092-d634b39f58a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:03.373701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24189d0c-2f95-42c4-8092-d634b39f58a2 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:03.373976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24189d0c-2f95-42c4-8092-d634b39f58a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:03.374247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24189d0c-2f95-42c4-8092-d634b39f58a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:03.374988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24189d0c-2f95-42c4-8092-d634b39f58a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:03.375291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24189d0c-2f95-42c4-8092-d634b39f58a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:03.376793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24189d0c-2f95-42c4-8092-d634b39f58a2 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:03.387099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 167/340] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:30:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:03.388405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fff7d4-4d18-4975-aa7f-fd3c1efcf5cb None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:03.388689 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fff7d4-4d18-4975-aa7f-fd3c1efcf5cb None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:03.388963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fff7d4-4d18-4975-aa7f-fd3c1efcf5cb None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:03.390690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 168/341] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:03.395377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fff7d4-4d18-4975-aa7f-fd3c1efcf5cb None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:03.538184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ee14e-c794-4c8e-85fa-56a9931be507 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:03.559188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:03.563853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fff7d4-4d18-4975-aa7f-fd3c1efcf5cb None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:03.582596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c3e25-a96f-4944-9ac3-9209c8237775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:03.583237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3e25-a96f-4944-9ac3-9209c8237775 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:03.583520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3e25-a96f-4944-9ac3-9209c8237775 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:03.583792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3e25-a96f-4944-9ac3-9209c8237775 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:03.605004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c3e25-a96f-4944-9ac3-9209c8237775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:03.607376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ae277-7e12-460b-a644-296c90dc7d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:03.607999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ae277-7e12-460b-a644-296c90dc7d1e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:03.608278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ae277-7e12-460b-a644-296c90dc7d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:03.608552 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ae277-7e12-460b-a644-296c90dc7d1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-cb6c3e25-a96f-4944-9ac3-9209c8237775 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:03.611897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:03.612756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:03.613545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:03.616447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 174/342] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:30:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:03.617847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3cafe-06e6-4772-9d85-da36ad84c2a4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:03.618342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3cafe-06e6-4772-9d85-da36ad84c2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:03.618844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3cafe-06e6-4772-9d85-da36ad84c2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:03.638202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0bb3cafe-06e6-4772-9d85-da36ad84c2a4 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:03.639858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0bb3cafe-06e6-4772-9d85-da36ad84c2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:03.658990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:03.687142 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ae277-7e12-460b-a644-296c90dc7d1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:03.688708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ae277-7e12-460b-a644-296c90dc7d1e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:03.726682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 175/343] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:03.728296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544eb23-69ca-4acd-b9c1-b618364f0595 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:03.728719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544eb23-69ca-4acd-b9c1-b618364f0595 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:03.729133 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544eb23-69ca-4acd-b9c1-b618364f0595 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:03.730984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 176/344] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:03.731823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544eb23-69ca-4acd-b9c1-b618364f0595 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:03.880365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:03.886850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 169/345] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:03.888360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4044c9-c235-429b-9916-001ef7138e01 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:03.888787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4044c9-c235-429b-9916-001ef7138e01 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:03.889226 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4044c9-c235-429b-9916-001ef7138e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:03.896514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544eb23-69ca-4acd-b9c1-b618364f0595 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:03.909555 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-aa4044c9-c235-429b-9916-001ef7138e01 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:03.911086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-aa4044c9-c235-429b-9916-001ef7138e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:03.990284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 170/346] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:03.991092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5af87-574c-433c-8c2c-7cc7b4d9cb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:03.991974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4044c9-c235-429b-9916-001ef7138e01 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:03.992090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5af87-574c-433c-8c2c-7cc7b4d9cb0b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:03.992288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5af87-574c-433c-8c2c-7cc7b4d9cb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:03.992560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5af87-574c-433c-8c2c-7cc7b4d9cb0b None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:03.993245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5af87-574c-433c-8c2c-7cc7b4d9cb0b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:03.993547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5af87-574c-433c-8c2c-7cc7b4d9cb0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:03.995033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5af87-574c-433c-8c2c-7cc7b4d9cb0b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:04.013935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 177/347] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:04.019121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5af87-574c-433c-8c2c-7cc7b4d9cb0b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:04.059904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:04.114848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4044c9-c235-429b-9916-001ef7138e01 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:04.148939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:04.156385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a389-c6bd-4be8-ad24-c998b2e12c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:04.157256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a389-c6bd-4be8-ad24-c998b2e12c7e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:04.157709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a389-c6bd-4be8-ad24-c998b2e12c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:04.158120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a389-c6bd-4be8-ad24-c998b2e12c7e None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:04.159576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24a389-c6bd-4be8-ad24-c998b2e12c7e None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:04.161188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24a389-c6bd-4be8-ad24-c998b2e12c7e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:04.164225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 171/348] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:03 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:04.171281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a389-c6bd-4be8-ad24-c998b2e12c7e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:04.276852 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a389-c6bd-4be8-ad24-c998b2e12c7e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:04.284825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5af87-574c-433c-8c2c-7cc7b4d9cb0b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:04.314358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410d7d6-f928-452a-ae5c-ac9aa57e8a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:04.315264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410d7d6-f928-452a-ae5c-ac9aa57e8a25 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:04.315711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410d7d6-f928-452a-ae5c-ac9aa57e8a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:04.316124 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410d7d6-f928-452a-ae5c-ac9aa57e8a25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:04.327997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410d7d6-f928-452a-ae5c-ac9aa57e8a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-7410d7d6-f928-452a-ae5c-ac9aa57e8a25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:04.333549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:04.335900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:04.336744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:04.338687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 172/349] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:04.339648 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410d7d6-f928-452a-ae5c-ac9aa57e8a25 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:04.392731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff436af-5ea1-4ab8-bd6e-3b52deb31a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:04.393518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff436af-5ea1-4ab8-bd6e-3b52deb31a71 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:04.394938 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff436af-5ea1-4ab8-bd6e-3b52deb31a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:04.395324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff436af-5ea1-4ab8-bd6e-3b52deb31a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:04.407707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:04.461597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410d7d6-f928-452a-ae5c-ac9aa57e8a25 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:04.489939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff436af-5ea1-4ab8-bd6e-3b52deb31a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:04.493643 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff436af-5ea1-4ab8-bd6e-3b52deb31a71 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:04.505743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58980b-ee4e-4d34-9787-19516669d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:04.507575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58980b-ee4e-4d34-9787-19516669d252 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:04.507575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58980b-ee4e-4d34-9787-19516669d252 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:04.507575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58980b-ee4e-4d34-9787-19516669d252 None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:04.518987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58980b-ee4e-4d34-9787-19516669d252 None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:04.520240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:04.520359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58980b-ee4e-4d34-9787-19516669d252 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:04.523731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 173/350] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:04 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:04.524756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58980b-ee4e-4d34-9787-19516669d252 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:04.625533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 178/351] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 23:30:04.626454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff436af-5ea1-4ab8-bd6e-3b52deb31a71 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:04.654841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58980b-ee4e-4d34-9787-19516669d252 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:04.686069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:04.699596 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef69e1a-6da8-4713-a119-97dae957cdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:04.700210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef69e1a-6da8-4713-a119-97dae957cdc9 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:04.703403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef69e1a-6da8-4713-a119-97dae957cdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:04.703403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef69e1a-6da8-4713-a119-97dae957cdc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:04.703403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef69e1a-6da8-4713-a119-97dae957cdc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:04.703403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef69e1a-6da8-4713-a119-97dae957cdc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:04.703403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef69e1a-6da8-4713-a119-97dae957cdc9 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:04.712756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 174/352] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:30:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:04.713617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef69e1a-6da8-4713-a119-97dae957cdc9 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:04.757384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff436af-5ea1-4ab8-bd6e-3b52deb31a71 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:04.778026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:04.813918 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f621dc-f8cb-45dc-90ce-cf68f3ea9abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:04.815120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f621dc-f8cb-45dc-90ce-cf68f3ea9abe None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:04.815543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f621dc-f8cb-45dc-90ce-cf68f3ea9abe None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:04.815955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f621dc-f8cb-45dc-90ce-cf68f3ea9abe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:04.832202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f621dc-f8cb-45dc-90ce-cf68f3ea9abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-19f621dc-f8cb-45dc-90ce-cf68f3ea9abe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:04.839470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:04.840585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:04.841542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 179/353] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:04.842271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aad9e-5533-4e27-914f-00659397e117 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:04.842271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aad9e-5533-4e27-914f-00659397e117 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:04.842585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aad9e-5533-4e27-914f-00659397e117 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:04.844724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 180/354] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:04.845468 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08aad9e-5533-4e27-914f-00659397e117 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:04.862724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef69e1a-6da8-4713-a119-97dae957cdc9 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:04.910348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d2aeb-fb8f-4670-a4b9-9a1478c9a901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:04.911136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d2aeb-fb8f-4670-a4b9-9a1478c9a901 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:04.911468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d2aeb-fb8f-4670-a4b9-9a1478c9a901 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:04.911752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d2aeb-fb8f-4670-a4b9-9a1478c9a901 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:04.913145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d2aeb-fb8f-4670-a4b9-9a1478c9a901 None admin] RBAC: Adding query filter params (group.id=7bb83bf8d47b4883ad7048ffdd699359, role.id=4d98a2b16f8f42bca3e5dac0374cdb54, scope.project.id=290b724bdca948a29018c8ecc35d1c8b) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:04.913511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d2aeb-fb8f-4670-a4b9-9a1478c9a901 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:04.915723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d2aeb-fb8f-4670-a4b9-9a1478c9a901 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:04.925813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 175/355] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Aug 29 23:30:04 2022] GET /identity/v3/role_assignments?group.id=7bb83bf8d47b4883ad7048ffdd699359&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:04.925813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d2aeb-fb8f-4670-a4b9-9a1478c9a901 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:04.981054 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08aad9e-5533-4e27-914f-00659397e117 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:04.992619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:05.025619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132411e-5915-40a1-b0a5-9376ca7deb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:05.026678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132411e-5915-40a1-b0a5-9376ca7deb6f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:05.027162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132411e-5915-40a1-b0a5-9376ca7deb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:05.027701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132411e-5915-40a1-b0a5-9376ca7deb6f None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:05.029218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132411e-5915-40a1-b0a5-9376ca7deb6f None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:05.031083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132411e-5915-40a1-b0a5-9376ca7deb6f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:05.033148 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 181/356] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:04 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:05.034330 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132411e-5915-40a1-b0a5-9376ca7deb6f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:05.048383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d2aeb-fb8f-4670-a4b9-9a1478c9a901 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:05.086846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb877444-8105-45aa-a641-59d45f53fa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:05.087632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb877444-8105-45aa-a641-59d45f53fa14 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:05.088197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:05.088971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb877444-8105-45aa-a641-59d45f53fa14 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:05.089567 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb877444-8105-45aa-a641-59d45f53fa14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:05.161922 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132411e-5915-40a1-b0a5-9376ca7deb6f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:05.177808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb877444-8105-45aa-a641-59d45f53fa14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:05.180142 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb877444-8105-45aa-a641-59d45f53fa14 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:05.212103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0c994-c17d-4c22-a7a1-9d528333e684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:05.212965 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0c994-c17d-4c22-a7a1-9d528333e684 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:05.213719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0c994-c17d-4c22-a7a1-9d528333e684 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:05.214163 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0c994-c17d-4c22-a7a1-9d528333e684 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:05.215124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0c994-c17d-4c22-a7a1-9d528333e684 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:05.215867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0c994-c17d-4c22-a7a1-9d528333e684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:05.217934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0c994-c17d-4c22-a7a1-9d528333e684 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:05.229878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 182/357] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:30:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:05.232043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b359d-bd3b-457c-b96a-7faaeda97964 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:05.232478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b359d-bd3b-457c-b96a-7faaeda97964 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:05.232893 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b359d-bd3b-457c-b96a-7faaeda97964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:05.247287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:05.258071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c39b359d-bd3b-457c-b96a-7faaeda97964 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:05.259708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c39b359d-bd3b-457c-b96a-7faaeda97964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:05.310575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 176/358] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:05.311574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb877444-8105-45aa-a641-59d45f53fa14 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:05.361107 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 183/359] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:05.362552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fbd2e-0c37-43bc-8d92-9e662f6fde19 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:05.362838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fbd2e-0c37-43bc-8d92-9e662f6fde19 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:05.363108 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fbd2e-0c37-43bc-8d92-9e662f6fde19 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:05.364367 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 184/360] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:05.365126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fbd2e-0c37-43bc-8d92-9e662f6fde19 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:05.418448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:05.442461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb877444-8105-45aa-a641-59d45f53fa14 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:05.480974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a226c-77ce-4701-b7a3-7140911a5fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:05.481620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a226c-77ce-4701-b7a3-7140911a5fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:05.481988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a226c-77ce-4701-b7a3-7140911a5fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:05.483288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a226c-77ce-4701-b7a3-7140911a5fa0 None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:05.483397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a226c-77ce-4701-b7a3-7140911a5fa0 None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:05.484855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a226c-77ce-4701-b7a3-7140911a5fa0 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:05.486471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 177/361] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:05 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:05.487422 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a226c-77ce-4701-b7a3-7140911a5fa0 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:05.495952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fbd2e-0c37-43bc-8d92-9e662f6fde19 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:05.507615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:05.534538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e54b4-f8a8-4965-913c-dead232d4b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:05.535737 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e54b4-f8a8-4965-913c-dead232d4b92 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:05.536230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e54b4-f8a8-4965-913c-dead232d4b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:05.536684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e54b4-f8a8-4965-913c-dead232d4b92 None admin] PATH_INFO: `/v3/users/c7ae99a1cf124aa781eda159f3c6c470` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:05.539227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e54b4-f8a8-4965-913c-dead232d4b92 None admin] RBAC: Authorizing `identity:get_user(user_id=c7ae99a1cf124aa781eda159f3c6c470)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:05.540970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e54b4-f8a8-4965-913c-dead232d4b92 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:05.542868 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 185/362] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:30:05 2022] GET /identity/v3/users/c7ae99a1cf124aa781eda159f3c6c470 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:05.544299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77861e3-4f82-4ef7-acc2-de3514b3cfa1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:05.549577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77861e3-4f82-4ef7-acc2-de3514b3cfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:05.550229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77861e3-4f82-4ef7-acc2-de3514b3cfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:05.573305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c77861e3-4f82-4ef7-acc2-de3514b3cfa1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:05.574847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c77861e3-4f82-4ef7-acc2-de3514b3cfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:05.602665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a226c-77ce-4701-b7a3-7140911a5fa0 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:05.641153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5181bd4-28d6-4d95-b1b6-af57b99ecdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:05.641651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5181bd4-28d6-4d95-b1b6-af57b99ecdee None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:05.641920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5181bd4-28d6-4d95-b1b6-af57b99ecdee None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:05.642193 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5181bd4-28d6-4d95-b1b6-af57b99ecdee None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:05.643397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5181bd4-28d6-4d95-b1b6-af57b99ecdee None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:05.644860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5181bd4-28d6-4d95-b1b6-af57b99ecdee None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:05.647032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 178/363] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:05 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:05.648084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5181bd4-28d6-4d95-b1b6-af57b99ecdee None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:05.655265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 186/364] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:05.656445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77861e3-4f82-4ef7-acc2-de3514b3cfa1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:05.691784 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:05.760077 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:05.780522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5181bd4-28d6-4d95-b1b6-af57b99ecdee None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:05.784553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77861e3-4f82-4ef7-acc2-de3514b3cfa1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:05.830705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4f766-2b75-48d6-af32-d2072e435c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:05.831670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4f766-2b75-48d6-af32-d2072e435c45 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:05.832154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4f766-2b75-48d6-af32-d2072e435c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:05.832649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4f766-2b75-48d6-af32-d2072e435c45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:05.833702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4f766-2b75-48d6-af32-d2072e435c45 None admin] RBAC: Adding query filter params (group.id=7bb83bf8d47b4883ad7048ffdd699359, role.id=a38face2eeae40c28edda83a4d46614f, scope.project.id=290b724bdca948a29018c8ecc35d1c8b) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:05.835412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4f766-2b75-48d6-af32-d2072e435c45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:05.837251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4f766-2b75-48d6-af32-d2072e435c45 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:05.837562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac29878-71a0-4092-b37d-cf6bfa543de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:05.838178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac29878-71a0-4092-b37d-cf6bfa543de9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:05.838474 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac29878-71a0-4092-b37d-cf6bfa543de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:05.838752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac29878-71a0-4092-b37d-cf6bfa543de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:05.839480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac29878-71a0-4092-b37d-cf6bfa543de9 None admin] RBAC: Adding query filter params (role.id=6f434b187a404b7cb1e9bb8303ce59b5, user.id=c7ae99a1cf124aa781eda159f3c6c470, scope.system=all) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:05.839787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac29878-71a0-4092-b37d-cf6bfa543de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:05.841565 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac29878-71a0-4092-b37d-cf6bfa543de9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:05.846966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 179/365] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Aug 29 23:30:05 2022] GET /identity/v3/role_assignments?group.id=7bb83bf8d47b4883ad7048ffdd699359&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=a38face2eeae40c28edda83a4d46614f => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:05.848564 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4f766-2b75-48d6-af32-d2072e435c45 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:05.858397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 187/366] 10.222.0.58 () {56 vars in 1308 bytes} [Mon Aug 29 23:30:05 2022] GET /identity/v3/role_assignments?user.id=c7ae99a1cf124aa781eda159f3c6c470&scope.system=all&role.id=6f434b187a404b7cb1e9bb8303ce59b5 => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:05.859490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac29878-71a0-4092-b37d-cf6bfa543de9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:05.996770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:06.011735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac29878-71a0-4092-b37d-cf6bfa543de9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:06.061299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096e04-318c-4b0a-ab9e-b6e1d180acb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:06.061959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096e04-318c-4b0a-ab9e-b6e1d180acb5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:06.062239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096e04-318c-4b0a-ab9e-b6e1d180acb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:06.062543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096e04-318c-4b0a-ab9e-b6e1d180acb5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:06.078147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b096e04-318c-4b0a-ab9e-b6e1d180acb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-7b096e04-318c-4b0a-ab9e-b6e1d180acb5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:06.085607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:06.086757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:06.087566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 188/367] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:30:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:06.088234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096e04-318c-4b0a-ab9e-b6e1d180acb5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:06.090668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4f766-2b75-48d6-af32-d2072e435c45 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:06.153132 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00472a-88da-486d-9135-8defeccf6f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:06.155609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00472a-88da-486d-9135-8defeccf6f13 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:06.155609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00472a-88da-486d-9135-8defeccf6f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:06.155609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00472a-88da-486d-9135-8defeccf6f13 None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:06.156614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00472a-88da-486d-9135-8defeccf6f13 None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:06.157765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00472a-88da-486d-9135-8defeccf6f13 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:06.159395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 180/368] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:05 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:06.163502 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00472a-88da-486d-9135-8defeccf6f13 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:06.208556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:06.257266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096e04-318c-4b0a-ab9e-b6e1d180acb5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:06.304889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f9417-08b9-4477-94be-7f50a5627d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:06.305455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9417-08b9-4477-94be-7f50a5627d75 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:06.305730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9417-08b9-4477-94be-7f50a5627d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:06.306000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9417-08b9-4477-94be-7f50a5627d75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:06.325485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f9417-08b9-4477-94be-7f50a5627d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d80f9417-08b9-4477-94be-7f50a5627d75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:06.331960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:06.332886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 189/369] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:30:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:06.334480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f9417-08b9-4477-94be-7f50a5627d75 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:06.353382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:06.357474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00472a-88da-486d-9135-8defeccf6f13 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:06.399964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62718b8e-dba2-448b-b5bc-c42da9b27cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:06.400791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62718b8e-dba2-448b-b5bc-c42da9b27cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:06.400997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62718b8e-dba2-448b-b5bc-c42da9b27cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:06.401214 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62718b8e-dba2-448b-b5bc-c42da9b27cd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:06.401887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62718b8e-dba2-448b-b5bc-c42da9b27cd1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:06.402186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62718b8e-dba2-448b-b5bc-c42da9b27cd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:06.405111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62718b8e-dba2-448b-b5bc-c42da9b27cd1 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:06.415688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 181/370] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:06.416389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62718b8e-dba2-448b-b5bc-c42da9b27cd1 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:06.478546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f9417-08b9-4477-94be-7f50a5627d75 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:06.514823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:06.536918 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9356743-e744-469b-b176-7d9cfe5d7b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:06.538858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356743-e744-469b-b176-7d9cfe5d7b35 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:06.539395 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356743-e744-469b-b176-7d9cfe5d7b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:06.539815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356743-e744-469b-b176-7d9cfe5d7b35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:06.540870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9356743-e744-469b-b176-7d9cfe5d7b35 None admin] RBAC: Adding query filter params (group.id=7bb83bf8d47b4883ad7048ffdd699359, role.id=4d98a2b16f8f42bca3e5dac0374cdb54, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:06.541619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9356743-e744-469b-b176-7d9cfe5d7b35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:06.543566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9356743-e744-469b-b176-7d9cfe5d7b35 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:06.544447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62718b8e-dba2-448b-b5bc-c42da9b27cd1 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:06.552398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 190/371] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Aug 29 23:30:06 2022] GET /identity/v3/role_assignments?group.id=7bb83bf8d47b4883ad7048ffdd699359&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:06.555999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9356743-e744-469b-b176-7d9cfe5d7b35 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:06.699333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed2a27-cfa1-41fa-bffb-004385c99fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:06.700032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed2a27-cfa1-41fa-bffb-004385c99fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:06.700341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed2a27-cfa1-41fa-bffb-004385c99fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:06.700617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed2a27-cfa1-41fa-bffb-004385c99fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:06.701509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed2a27-cfa1-41fa-bffb-004385c99fc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:06.701817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed2a27-cfa1-41fa-bffb-004385c99fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:06.708059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed2a27-cfa1-41fa-bffb-004385c99fc4 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:06.739417 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 182/372] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:06.748230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed2a27-cfa1-41fa-bffb-004385c99fc4 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:06.762450 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:06.809498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9356743-e744-469b-b176-7d9cfe5d7b35 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:06.895218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7da580-be22-453c-a5ee-596b101446d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:06.896102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7da580-be22-453c-a5ee-596b101446d7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:06.896376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7da580-be22-453c-a5ee-596b101446d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:06.896836 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7da580-be22-453c-a5ee-596b101446d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:06.986099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed2a27-cfa1-41fa-bffb-004385c99fc4 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:07.047473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7da580-be22-453c-a5ee-596b101446d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:07.051012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7da580-be22-453c-a5ee-596b101446d7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:07.094744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f8d5b-bbd8-4660-b08f-9ad6031b9b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:07.095709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f8d5b-bbd8-4660-b08f-9ad6031b9b07 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:07.096181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f8d5b-bbd8-4660-b08f-9ad6031b9b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:07.096669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f8d5b-bbd8-4660-b08f-9ad6031b9b07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:07.122930 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:07.242787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:07.257632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f8d5b-bbd8-4660-b08f-9ad6031b9b07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:07.261294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f8d5b-bbd8-4660-b08f-9ad6031b9b07 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:07.351060 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 191/373] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 796 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:07.361074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7da580-be22-453c-a5ee-596b101446d7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:07.598635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 183/374] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 851 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 23:30:07.601396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed897c8-eabd-4c4c-9560-4b6d697fa744 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:07.601925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed897c8-eabd-4c4c-9560-4b6d697fa744 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:07.602474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed897c8-eabd-4c4c-9560-4b6d697fa744 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:07.605382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7da580-be22-453c-a5ee-596b101446d7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:07.606937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 184/375] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:07.608043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed897c8-eabd-4c4c-9560-4b6d697fa744 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:07.643438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207dd02-fb82-4afe-8505-a4ce91985b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:07.643679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207dd02-fb82-4afe-8505-a4ce91985b46 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:07.643902 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207dd02-fb82-4afe-8505-a4ce91985b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:07.644092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207dd02-fb82-4afe-8505-a4ce91985b46 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:07.645289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207dd02-fb82-4afe-8505-a4ce91985b46 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:07.647400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207dd02-fb82-4afe-8505-a4ce91985b46 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:07.649651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 192/376] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:07 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:07.650875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c354b76-7509-4340-addc-93198f18d49c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:07.651153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c354b76-7509-4340-addc-93198f18d49c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:07.651423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c354b76-7509-4340-addc-93198f18d49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:07.666311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:07.672622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6c354b76-7509-4340-addc-93198f18d49c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:07.673768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6c354b76-7509-4340-addc-93198f18d49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:07.719867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:07.742461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed897c8-eabd-4c4c-9560-4b6d697fa744 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:07.757824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 193/377] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:07.758744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c354b76-7509-4340-addc-93198f18d49c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:07.790308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e86739-5d20-4ca9-861c-ce2a17754b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:07.792149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e86739-5d20-4ca9-861c-ce2a17754b1c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:07.792893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e86739-5d20-4ca9-861c-ce2a17754b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:07.793638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e86739-5d20-4ca9-861c-ce2a17754b1c None admin] PATH_INFO: `/v3/projects/291cfd89a4d34a9da9d99e5ca0c9d4dd` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:07.795858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e86739-5d20-4ca9-861c-ce2a17754b1c None admin] RBAC: Authorizing `identity:get_project(project_id=291cfd89a4d34a9da9d99e5ca0c9d4dd)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:07.797662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e86739-5d20-4ca9-861c-ce2a17754b1c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:07.799967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 185/378] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:07 2022] GET /identity/v3/projects/291cfd89a4d34a9da9d99e5ca0c9d4dd => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:07.806365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e86739-5d20-4ca9-861c-ce2a17754b1c None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:07.934069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c354b76-7509-4340-addc-93198f18d49c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:07.983874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e86739-5d20-4ca9-861c-ce2a17754b1c None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:08.005289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:08.006471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160895c-440a-4171-b204-a57de4d4079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:08.007629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160895c-440a-4171-b204-a57de4d4079b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:08.007629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160895c-440a-4171-b204-a57de4d4079b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:08.007629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160895c-440a-4171-b204-a57de4d4079b None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:08.008800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160895c-440a-4171-b204-a57de4d4079b None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:08.011040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160895c-440a-4171-b204-a57de4d4079b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:08.013110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 194/379] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:07 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:08.018813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160895c-440a-4171-b204-a57de4d4079b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:08.067227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6196d74-dbf2-46c2-aabf-b179b6975b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:08.067874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6196d74-dbf2-46c2-aabf-b179b6975b7d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:08.068156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6196d74-dbf2-46c2-aabf-b179b6975b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:08.068571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6196d74-dbf2-46c2-aabf-b179b6975b7d None admin] PATH_INFO: `/v3/groups/ce6e0c80918444debacfb0d6b84bf680` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:08.090839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:08.095181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6196d74-dbf2-46c2-aabf-b179b6975b7d None admin] RBAC: Authorizing `identity:get_group(group_id=ce6e0c80918444debacfb0d6b84bf680)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:08.096667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6196d74-dbf2-46c2-aabf-b179b6975b7d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:08.098389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 186/380] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:07 2022] GET /identity/v3/groups/ce6e0c80918444debacfb0d6b84bf680 => generated 229 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:08.107482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6196d74-dbf2-46c2-aabf-b179b6975b7d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:08.160176 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160895c-440a-4171-b204-a57de4d4079b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:08.200259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a1c5a-c99c-45af-a770-473ad35942ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:08.200845 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a1c5a-c99c-45af-a770-473ad35942ce None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:08.201130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a1c5a-c99c-45af-a770-473ad35942ce None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:08.201406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a1c5a-c99c-45af-a770-473ad35942ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:08.202253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a1c5a-c99c-45af-a770-473ad35942ce None admin] RBAC: Adding query filter params (group.id=7bb83bf8d47b4883ad7048ffdd699359, role.id=a38face2eeae40c28edda83a4d46614f, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:08.202593 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a1c5a-c99c-45af-a770-473ad35942ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:08.204249 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a1c5a-c99c-45af-a770-473ad35942ce None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:08.211911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 195/381] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Aug 29 23:30:08 2022] GET /identity/v3/role_assignments?group.id=7bb83bf8d47b4883ad7048ffdd699359&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=a38face2eeae40c28edda83a4d46614f => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:08.213177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a1c5a-c99c-45af-a770-473ad35942ce None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:08.243206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6196d74-dbf2-46c2-aabf-b179b6975b7d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:08.289508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c25b5d-8e52-4fb5-a1b7-22e4e6ae84b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:08.290122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c25b5d-8e52-4fb5-a1b7-22e4e6ae84b3 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:08.290467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c25b5d-8e52-4fb5-a1b7-22e4e6ae84b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:08.290844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c25b5d-8e52-4fb5-a1b7-22e4e6ae84b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:08.291906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c25b5d-8e52-4fb5-a1b7-22e4e6ae84b3 None admin] RBAC: Adding query filter params (group.id=ce6e0c80918444debacfb0d6b84bf680, role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=291cfd89a4d34a9da9d99e5ca0c9d4dd) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:08.292356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c25b5d-8e52-4fb5-a1b7-22e4e6ae84b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:08.294593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c25b5d-8e52-4fb5-a1b7-22e4e6ae84b3 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:08.301790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 187/382] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Aug 29 23:30:08 2022] GET /identity/v3/role_assignments?group.id=ce6e0c80918444debacfb0d6b84bf680&scope.project.id=291cfd89a4d34a9da9d99e5ca0c9d4dd&role.id=d86dba8efe20423abdde06031690beb8 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:08.359708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a1c5a-c99c-45af-a770-473ad35942ce None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:08.368632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:08.401354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95287054-0dc2-4494-bc89-555d3acbb119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:08.402059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95287054-0dc2-4494-bc89-555d3acbb119 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:08.402341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95287054-0dc2-4494-bc89-555d3acbb119 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:08.402639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95287054-0dc2-4494-bc89-555d3acbb119 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:08.417123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95287054-0dc2-4494-bc89-555d3acbb119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-95287054-0dc2-4494-bc89-555d3acbb119 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:08.423080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:08.424094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:08.425165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 196/383] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:08.432931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c25b5d-8e52-4fb5-a1b7-22e4e6ae84b3 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:08.488929 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:08.569916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c25b5d-8e52-4fb5-a1b7-22e4e6ae84b3 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:08.617320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d56683-1c1b-4cb3-9766-6cddfdb5cceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:08.618369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d56683-1c1b-4cb3-9766-6cddfdb5cceb None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:08.618891 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d56683-1c1b-4cb3-9766-6cddfdb5cceb None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:08.619463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d56683-1c1b-4cb3-9766-6cddfdb5cceb None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:08.620686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d56683-1c1b-4cb3-9766-6cddfdb5cceb None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:08.621196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d56683-1c1b-4cb3-9766-6cddfdb5cceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:08.623052 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d56683-1c1b-4cb3-9766-6cddfdb5cceb None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:08.637000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 188/384] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:30:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:08.643016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95287054-0dc2-4494-bc89-555d3acbb119 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:08.674987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207ea02-e173-41c3-ae8c-fa3ddf469a57 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:08.675422 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207ea02-e173-41c3-ae8c-fa3ddf469a57 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:08.675665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207ea02-e173-41c3-ae8c-fa3ddf469a57 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:08.677171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 189/385] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:30:08.684375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048879ad-09d9-4068-bc36-46de58b2c3c0 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:08.684696 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048879ad-09d9-4068-bc36-46de58b2c3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:08.684923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048879ad-09d9-4068-bc36-46de58b2c3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:08.711309 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-048879ad-09d9-4068-bc36-46de58b2c3c0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:08.712539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-048879ad-09d9-4068-bc36-46de58b2c3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:08.776954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95287054-0dc2-4494-bc89-555d3acbb119 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:08.818605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d12bf4-7e6a-4212-a67e-d85a3b91f1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:08.819251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d12bf4-7e6a-4212-a67e-d85a3b91f1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:08.819519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d12bf4-7e6a-4212-a67e-d85a3b91f1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:08.819897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d12bf4-7e6a-4212-a67e-d85a3b91f1c0 None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:08.821191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d12bf4-7e6a-4212-a67e-d85a3b91f1c0 None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:08.822717 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d12bf4-7e6a-4212-a67e-d85a3b91f1c0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:08.824802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 197/386] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:08 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:08.830249 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d12bf4-7e6a-4212-a67e-d85a3b91f1c0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:08.845588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 190/387] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:08.870874 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:08.940898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d12bf4-7e6a-4212-a67e-d85a3b91f1c0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:08.978219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15108ace-4143-43c8-be7e-046f4d4916fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:08.978899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15108ace-4143-43c8-be7e-046f4d4916fc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:08.979174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15108ace-4143-43c8-be7e-046f4d4916fc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:08.979440 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15108ace-4143-43c8-be7e-046f4d4916fc None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:08.980610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15108ace-4143-43c8-be7e-046f4d4916fc None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:08.982061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15108ace-4143-43c8-be7e-046f4d4916fc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:08.984060 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 198/388] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:08 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:08.990605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048879ad-09d9-4068-bc36-46de58b2c3c0 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:09.042173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:09.154402 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048879ad-09d9-4068-bc36-46de58b2c3c0 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:09.196826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec9185-1002-4557-a439-5a2c08f392ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:09.197504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec9185-1002-4557-a439-5a2c08f392ce None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:09.197790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec9185-1002-4557-a439-5a2c08f392ce None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:09.198071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec9185-1002-4557-a439-5a2c08f392ce None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b/groups/7bb83bf8d47b4883ad7048ffdd699359/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:09.204025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec9185-1002-4557-a439-5a2c08f392ce None admin] RBAC: Authorizing `identity:create_grant(project_id=290b724bdca948a29018c8ecc35d1c8b, group_id=7bb83bf8d47b4883ad7048ffdd699359, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:09.205495 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec9185-1002-4557-a439-5a2c08f392ce None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:09.278398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 191/389] 10.222.0.58 () {58 vars in 1442 bytes} [Mon Aug 29 23:30:08 2022] PUT /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b/groups/7bb83bf8d47b4883ad7048ffdd699359/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:30:09.403878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15108ace-4143-43c8-be7e-046f4d4916fc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:09.526749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:09.531753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15108ace-4143-43c8-be7e-046f4d4916fc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:09.571081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2747b8-12c6-4b5e-885a-27e9af6ba6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:09.571716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2747b8-12c6-4b5e-885a-27e9af6ba6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:09.571993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2747b8-12c6-4b5e-885a-27e9af6ba6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:09.572259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2747b8-12c6-4b5e-885a-27e9af6ba6c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:09.585455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2747b8-12c6-4b5e-885a-27e9af6ba6c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-2a2747b8-12c6-4b5e-885a-27e9af6ba6c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:09.591230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:09.592230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:09.593357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 199/390] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:30:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:09.600789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec9185-1002-4557-a439-5a2c08f392ce None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:09.624478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81b4e-af07-48d9-a5a5-cbfc1387ed88 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:09.624688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81b4e-af07-48d9-a5a5-cbfc1387ed88 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:09.624901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81b4e-af07-48d9-a5a5-cbfc1387ed88 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:09.626787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 200/391] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:09.632999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9a4f0-20d6-40e3-955b-98e2c5830178 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:09.633338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9a4f0-20d6-40e3-955b-98e2c5830178 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:09.633615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9a4f0-20d6-40e3-955b-98e2c5830178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:09.657449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-29b9a4f0-20d6-40e3-955b-98e2c5830178 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:09.658806 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-29b9a4f0-20d6-40e3-955b-98e2c5830178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:09.772401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 201/392] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:30:09.784814 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:09.801386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec9185-1002-4557-a439-5a2c08f392ce None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:09.876394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897ef83-dcb5-4d8d-819b-23e02b4ad6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:09.876927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897ef83-dcb5-4d8d-819b-23e02b4ad6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:09.877207 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897ef83-dcb5-4d8d-819b-23e02b4ad6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:09.877478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897ef83-dcb5-4d8d-819b-23e02b4ad6c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:09.878151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f897ef83-dcb5-4d8d-819b-23e02b4ad6c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:09.878462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f897ef83-dcb5-4d8d-819b-23e02b4ad6c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:09.879834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f897ef83-dcb5-4d8d-819b-23e02b4ad6c0 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:09.892483 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 192/393] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:09.898851 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9a4f0-20d6-40e3-955b-98e2c5830178 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:10.016697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:10.028795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9a4f0-20d6-40e3-955b-98e2c5830178 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:10.074681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ef296-3dc5-4576-a515-b738b6c19bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:10.075371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ef296-3dc5-4576-a515-b738b6c19bde None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:10.075652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ef296-3dc5-4576-a515-b738b6c19bde None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:10.075928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ef296-3dc5-4576-a515-b738b6c19bde None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:10.077073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ef296-3dc5-4576-a515-b738b6c19bde None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:10.078732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ef296-3dc5-4576-a515-b738b6c19bde None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:10.081338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 202/394] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:09 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:10.087243 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897ef83-dcb5-4d8d-819b-23e02b4ad6c0 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:10.229370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897ef83-dcb5-4d8d-819b-23e02b4ad6c0 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:10.279885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99ae74-07fb-41a8-8eda-cf96eafbb6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:10.280532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99ae74-07fb-41a8-8eda-cf96eafbb6ed None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:10.280807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99ae74-07fb-41a8-8eda-cf96eafbb6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:10.281079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99ae74-07fb-41a8-8eda-cf96eafbb6ed None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:10.282235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e99ae74-07fb-41a8-8eda-cf96eafbb6ed None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:10.283688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e99ae74-07fb-41a8-8eda-cf96eafbb6ed None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:10.285617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 193/395] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:10 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:10.291248 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ef296-3dc5-4576-a515-b738b6c19bde None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:10.391673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99ae74-07fb-41a8-8eda-cf96eafbb6ed None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:10.435519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:10.493920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ef296-3dc5-4576-a515-b738b6c19bde None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:10.572747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99ae74-07fb-41a8-8eda-cf96eafbb6ed None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:10.579231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8b611-63a9-4f1b-87ef-2f80c5e46b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:10.579919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8b611-63a9-4f1b-87ef-2f80c5e46b1a None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:10.580195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8b611-63a9-4f1b-87ef-2f80c5e46b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:10.580466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8b611-63a9-4f1b-87ef-2f80c5e46b1a None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b/groups/7bb83bf8d47b4883ad7048ffdd699359/roles/a38face2eeae40c28edda83a4d46614f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:10.582821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8b611-63a9-4f1b-87ef-2f80c5e46b1a None admin] RBAC: Authorizing `identity:create_grant(project_id=290b724bdca948a29018c8ecc35d1c8b, group_id=7bb83bf8d47b4883ad7048ffdd699359, role_id=a38face2eeae40c28edda83a4d46614f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:10.584895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8b611-63a9-4f1b-87ef-2f80c5e46b1a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:10.615471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8609724-c472-4627-ad84-f8e65adc5fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:10.616118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8609724-c472-4627-ad84-f8e65adc5fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:10.616461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8609724-c472-4627-ad84-f8e65adc5fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:10.616740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8609724-c472-4627-ad84-f8e65adc5fe0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:10.633072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8609724-c472-4627-ad84-f8e65adc5fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-b8609724-c472-4627-ad84-f8e65adc5fe0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:10.640017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:10.641117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:10.641935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 194/396] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:10.649209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8609724-c472-4627-ad84-f8e65adc5fe0 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:10.669018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 203/397] 10.222.0.58 () {58 vars in 1442 bytes} [Mon Aug 29 23:30:10 2022] PUT /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b/groups/7bb83bf8d47b4883ad7048ffdd699359/roles/a38face2eeae40c28edda83a4d46614f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:30:10.752978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:10.779616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8609724-c472-4627-ad84-f8e65adc5fe0 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:10.826154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d71d6-c41c-4cd5-8b4a-d8380aa5917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:10.827504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d71d6-c41c-4cd5-8b4a-d8380aa5917c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:10.827936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d71d6-c41c-4cd5-8b4a-d8380aa5917c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:10.828349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d71d6-c41c-4cd5-8b4a-d8380aa5917c None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:10.829175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d71d6-c41c-4cd5-8b4a-d8380aa5917c None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:10.829616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d71d6-c41c-4cd5-8b4a-d8380aa5917c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:10.831259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d71d6-c41c-4cd5-8b4a-d8380aa5917c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:10.842482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 195/398] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:30:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:10.847958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8b611-63a9-4f1b-87ef-2f80c5e46b1a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:10.872391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa85ad-9086-4668-afb7-2e4b73e13ff4 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:10.872901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa85ad-9086-4668-afb7-2e4b73e13ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:10.873345 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa85ad-9086-4668-afb7-2e4b73e13ff4 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:10.875194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 196/399] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:30:10.882203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ce271-acfb-4c4d-9e53-62b02fbdf7f1 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:10.882766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ce271-acfb-4c4d-9e53-62b02fbdf7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:10.883191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ce271-acfb-4c4d-9e53-62b02fbdf7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:10.883746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:10.926100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-da9ce271-acfb-4c4d-9e53-62b02fbdf7f1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:10.929370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-da9ce271-acfb-4c4d-9e53-62b02fbdf7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:11.021506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8b611-63a9-4f1b-87ef-2f80c5e46b1a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:11.033666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 197/400] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:30:11.035394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1670d6-085f-47e5-b136-8913d34fff53 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:11.035940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1670d6-085f-47e5-b136-8913d34fff53 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:11.036423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1670d6-085f-47e5-b136-8913d34fff53 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:11.037943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 198/401] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:11.043714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68946534-669b-4257-8d71-3e6ac85f7b71 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:11.044331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68946534-669b-4257-8d71-3e6ac85f7b71 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:11.044822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68946534-669b-4257-8d71-3e6ac85f7b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:11.067231 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-68946534-669b-4257-8d71-3e6ac85f7b71 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:11.068828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-68946534-669b-4257-8d71-3e6ac85f7b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:11.086535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da457f9-6ea0-4019-8c24-8d4bc9ee8e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:11.087241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da457f9-6ea0-4019-8c24-8d4bc9ee8e31 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:11.087693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da457f9-6ea0-4019-8c24-8d4bc9ee8e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:11.088147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da457f9-6ea0-4019-8c24-8d4bc9ee8e31 None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:11.089357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da457f9-6ea0-4019-8c24-8d4bc9ee8e31 None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:11.090865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da457f9-6ea0-4019-8c24-8d4bc9ee8e31 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:11.095266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 204/402] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:10 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:11.101426 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da457f9-6ea0-4019-8c24-8d4bc9ee8e31 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:11.209953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:11.232160 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 199/403] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:30:11.250820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:11.260774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da457f9-6ea0-4019-8c24-8d4bc9ee8e31 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:11.326214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba572516-aa08-4f79-8f60-eacffcc686a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:11.326870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba572516-aa08-4f79-8f60-eacffcc686a3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:11.327263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba572516-aa08-4f79-8f60-eacffcc686a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:11.327547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba572516-aa08-4f79-8f60-eacffcc686a3 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:11.328773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba572516-aa08-4f79-8f60-eacffcc686a3 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:11.330415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba572516-aa08-4f79-8f60-eacffcc686a3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:11.332601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 205/404] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:11 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:11.338858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68946534-669b-4257-8d71-3e6ac85f7b71 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:11.359584 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba572516-aa08-4f79-8f60-eacffcc686a3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:11.509687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba572516-aa08-4f79-8f60-eacffcc686a3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:11.547630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924527d0-44a2-48f2-978e-a1f3f7018ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:11.548305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924527d0-44a2-48f2-978e-a1f3f7018ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:11.548639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924527d0-44a2-48f2-978e-a1f3f7018ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:11.548843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924527d0-44a2-48f2-978e-a1f3f7018ae6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:11.557768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68946534-669b-4257-8d71-3e6ac85f7b71 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:11.560389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924527d0-44a2-48f2-978e-a1f3f7018ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-924527d0-44a2-48f2-978e-a1f3f7018ae6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:11.565290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:11.566221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:11.567103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 206/405] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:30:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:11.567757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924527d0-44a2-48f2-978e-a1f3f7018ae6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:11.589724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:11.603115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff4c12-fb46-4ae4-ac4b-02758a843d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:11.604713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff4c12-fb46-4ae4-ac4b-02758a843d0b None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:11.605477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff4c12-fb46-4ae4-ac4b-02758a843d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:11.605962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff4c12-fb46-4ae4-ac4b-02758a843d0b None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/groups/7bb83bf8d47b4883ad7048ffdd699359/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:11.608889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff4c12-fb46-4ae4-ac4b-02758a843d0b None admin] RBAC: Authorizing `identity:create_grant(project_id=45aa2f6db17f4df1b7f892a95d31b8fc, group_id=7bb83bf8d47b4883ad7048ffdd699359, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:11.610779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff4c12-fb46-4ae4-ac4b-02758a843d0b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:11.683150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924527d0-44a2-48f2-978e-a1f3f7018ae6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:11.691742 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 200/406] 10.222.0.58 () {58 vars in 1442 bytes} [Mon Aug 29 23:30:11 2022] PUT /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/groups/7bb83bf8d47b4883ad7048ffdd699359/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:30:11.693613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff4c12-fb46-4ae4-ac4b-02758a843d0b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:11.724990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa3cba-e25b-4a17-a329-845731793762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:11.725958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa3cba-e25b-4a17-a329-845731793762 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:11.726398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa3cba-e25b-4a17-a329-845731793762 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:11.726886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa3cba-e25b-4a17-a329-845731793762 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:11.763831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa3cba-e25b-4a17-a329-845731793762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-fcaa3cba-e25b-4a17-a329-845731793762 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:11.769255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:11.770585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:11.771975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:11.784518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 207/407] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:30:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:11.790671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:11.792119 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa3cba-e25b-4a17-a329-845731793762 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:11.835586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff4c12-fb46-4ae4-ac4b-02758a843d0b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:11.884442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d5332-4611-4839-95c5-10b82705e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:11.885106 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d5332-4611-4839-95c5-10b82705e2dd None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:11.885554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d5332-4611-4839-95c5-10b82705e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:11.885834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d5332-4611-4839-95c5-10b82705e2dd None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:11.887414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d5332-4611-4839-95c5-10b82705e2dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:11.887875 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d5332-4611-4839-95c5-10b82705e2dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:11.889459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d5332-4611-4839-95c5-10b82705e2dd None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:11.901271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 201/408] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:11.907148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d5332-4611-4839-95c5-10b82705e2dd None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:11.930119 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:11.966347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa3cba-e25b-4a17-a329-845731793762 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:12.022367 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fcafc-511f-4814-b23e-98b8ff5e5046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:12.023729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fcafc-511f-4814-b23e-98b8ff5e5046 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:12.024019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fcafc-511f-4814-b23e-98b8ff5e5046 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:12.024297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fcafc-511f-4814-b23e-98b8ff5e5046 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:12.025254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fcafc-511f-4814-b23e-98b8ff5e5046 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:12.025982 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fcafc-511f-4814-b23e-98b8ff5e5046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:12.027427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fcafc-511f-4814-b23e-98b8ff5e5046 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:12.043488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 208/409] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:12.043685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d5332-4611-4839-95c5-10b82705e2dd None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:12.051723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fcafc-511f-4814-b23e-98b8ff5e5046 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:12.087773 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:12.100171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9866f2-49da-442f-afda-af9c39e55c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:12.100870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9866f2-49da-442f-afda-af9c39e55c8c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:12.101106 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9866f2-49da-442f-afda-af9c39e55c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:12.101464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9866f2-49da-442f-afda-af9c39e55c8c None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:12.102879 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9866f2-49da-442f-afda-af9c39e55c8c None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:12.104449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9866f2-49da-442f-afda-af9c39e55c8c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:12.110457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 202/410] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:11 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:12.110457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cbd0b-23b7-4a88-a56f-9c9cdda2e251 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:12.110457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cbd0b-23b7-4a88-a56f-9c9cdda2e251 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:12.110457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cbd0b-23b7-4a88-a56f-9c9cdda2e251 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:12.110457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 203/411] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:12.115111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cbd0b-23b7-4a88-a56f-9c9cdda2e251 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:12.199183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fcafc-511f-4814-b23e-98b8ff5e5046 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:12.239520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78d847-2c3e-441b-b10b-b699cbac9afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:12.240536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78d847-2c3e-441b-b10b-b699cbac9afb None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:12.240840 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78d847-2c3e-441b-b10b-b699cbac9afb None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:12.241114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78d847-2c3e-441b-b10b-b699cbac9afb None admin] PATH_INFO: `/v3/groups/ce6e0c80918444debacfb0d6b84bf680` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:12.242500 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78d847-2c3e-441b-b10b-b699cbac9afb None admin] RBAC: Authorizing `identity:get_group(group_id=ce6e0c80918444debacfb0d6b84bf680)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:12.244058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78d847-2c3e-441b-b10b-b699cbac9afb None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:12.250642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 209/412] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:12 2022] GET /identity/v3/groups/ce6e0c80918444debacfb0d6b84bf680 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:12.250924 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cbd0b-23b7-4a88-a56f-9c9cdda2e251 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:12.252063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4c6ed-6b2d-4d18-a0d5-7e198b6b9e9a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:12.252354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4c6ed-6b2d-4d18-a0d5-7e198b6b9e9a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:12.252634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4c6ed-6b2d-4d18-a0d5-7e198b6b9e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:12.261197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:12.275213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c9b4c6ed-6b2d-4d18-a0d5-7e198b6b9e9a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:12.276499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c9b4c6ed-6b2d-4d18-a0d5-7e198b6b9e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:12.298763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079228f-b5e8-40bb-a8b0-aa33c0145bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:12.299710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079228f-b5e8-40bb-a8b0-aa33c0145bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:12.300162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079228f-b5e8-40bb-a8b0-aa33c0145bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:12.301152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079228f-b5e8-40bb-a8b0-aa33c0145bd4 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:12.303371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079228f-b5e8-40bb-a8b0-aa33c0145bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:12.305581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079228f-b5e8-40bb-a8b0-aa33c0145bd4 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:12.313685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 204/413] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:12 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:12.313685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079228f-b5e8-40bb-a8b0-aa33c0145bd4 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:12.377046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 210/414] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:12.378659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4c6ed-6b2d-4d18-a0d5-7e198b6b9e9a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:12.403307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:12.452988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079228f-b5e8-40bb-a8b0-aa33c0145bd4 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:12.493893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dad25-6054-474a-a737-d00fb237596a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:12.494918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dad25-6054-474a-a737-d00fb237596a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:12.495864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dad25-6054-474a-a737-d00fb237596a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:12.496337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dad25-6054-474a-a737-d00fb237596a None admin] PATH_INFO: `/v3/projects/291cfd89a4d34a9da9d99e5ca0c9d4dd` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:12.497802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34dad25-6054-474a-a737-d00fb237596a None admin] RBAC: Authorizing `identity:get_project(project_id=291cfd89a4d34a9da9d99e5ca0c9d4dd)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:12.500592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34dad25-6054-474a-a737-d00fb237596a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:12.503434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 205/415] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:12 2022] GET /identity/v3/projects/291cfd89a4d34a9da9d99e5ca0c9d4dd => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:12.509935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dad25-6054-474a-a737-d00fb237596a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:12.637533 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dad25-6054-474a-a737-d00fb237596a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:12.658312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4c6ed-6b2d-4d18-a0d5-7e198b6b9e9a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:12.671418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:12.684288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba012879-2e1d-48d7-b54a-29c055b2f83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:12.685272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba012879-2e1d-48d7-b54a-29c055b2f83f None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:12.685741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba012879-2e1d-48d7-b54a-29c055b2f83f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:12.686371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba012879-2e1d-48d7-b54a-29c055b2f83f None admin] PATH_INFO: `/v3/projects/291cfd89a4d34a9da9d99e5ca0c9d4dd/groups/ce6e0c80918444debacfb0d6b84bf680/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:12.689206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba012879-2e1d-48d7-b54a-29c055b2f83f None admin] RBAC: Authorizing `identity:create_grant(project_id=291cfd89a4d34a9da9d99e5ca0c9d4dd, group_id=ce6e0c80918444debacfb0d6b84bf680, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:12.692911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba012879-2e1d-48d7-b54a-29c055b2f83f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:12.730969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a3420-6a13-4785-bb55-e09c6d998027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:12.732176 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a3420-6a13-4785-bb55-e09c6d998027 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:12.732582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a3420-6a13-4785-bb55-e09c6d998027 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:12.732976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a3420-6a13-4785-bb55-e09c6d998027 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/groups/7bb83bf8d47b4883ad7048ffdd699359/roles/a38face2eeae40c28edda83a4d46614f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:12.735405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a3420-6a13-4785-bb55-e09c6d998027 None admin] RBAC: Authorizing `identity:create_grant(project_id=45aa2f6db17f4df1b7f892a95d31b8fc, group_id=7bb83bf8d47b4883ad7048ffdd699359, role_id=a38face2eeae40c28edda83a4d46614f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:12.736965 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a3420-6a13-4785-bb55-e09c6d998027 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:12.741355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:12.780248 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 206/416] 10.222.0.58 () {58 vars in 1442 bytes} [Mon Aug 29 23:30:12 2022] PUT /identity/v3/projects/291cfd89a4d34a9da9d99e5ca0c9d4dd/groups/ce6e0c80918444debacfb0d6b84bf680/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:30:12.794195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba012879-2e1d-48d7-b54a-29c055b2f83f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:12.837670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 211/417] 10.222.0.58 () {58 vars in 1442 bytes} [Mon Aug 29 23:30:12 2022] PUT /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc/groups/7bb83bf8d47b4883ad7048ffdd699359/roles/a38face2eeae40c28edda83a4d46614f => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:30:12.945791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba012879-2e1d-48d7-b54a-29c055b2f83f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:13.004571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045c3c0-6154-49b4-9280-8ccfab0c66a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:13.006396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045c3c0-6154-49b4-9280-8ccfab0c66a5 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:13.006958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045c3c0-6154-49b4-9280-8ccfab0c66a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:13.007450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045c3c0-6154-49b4-9280-8ccfab0c66a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:13.034923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045c3c0-6154-49b4-9280-8ccfab0c66a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-d045c3c0-6154-49b4-9280-8ccfab0c66a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:13.042004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:13.043022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:13.043824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:13.044426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 207/418] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:13.050958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a3420-6a13-4785-bb55-e09c6d998027 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:13.188652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b0fac-2283-40c1-8741-e5eb3e0749a8 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:13.188965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b0fac-2283-40c1-8741-e5eb3e0749a8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:13.189212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b0fac-2283-40c1-8741-e5eb3e0749a8 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:13.190912 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 208/419] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:30:13.205609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a3420-6a13-4785-bb55-e09c6d998027 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:13.207759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0b34-c63e-46fe-b2d1-d6de130a92c9 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:13.208059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0b34-c63e-46fe-b2d1-d6de130a92c9 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:13.210261 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0b34-c63e-46fe-b2d1-d6de130a92c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:13.250473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-bf6c0b34-c63e-46fe-b2d1-d6de130a92c9 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:13.251690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bf6c0b34-c63e-46fe-b2d1-d6de130a92c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:13.308360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ad265-d5a5-482a-a3d2-d647a2e67021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:13.308903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ad265-d5a5-482a-a3d2-d647a2e67021 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:13.309180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ad265-d5a5-482a-a3d2-d647a2e67021 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:13.309453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ad265-d5a5-482a-a3d2-d647a2e67021 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:13.310162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ad265-d5a5-482a-a3d2-d647a2e67021 None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:13.316540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ad265-d5a5-482a-a3d2-d647a2e67021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:13.316540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ad265-d5a5-482a-a3d2-d647a2e67021 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:13.356073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 212/420] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:30:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:13.356073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ad265-d5a5-482a-a3d2-d647a2e67021 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:13.389047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:13.452826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 209/421] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:30:13.476879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:13.548831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ad265-d5a5-482a-a3d2-d647a2e67021 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:13.634728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad2375-f6a8-45e3-875d-99c5690afda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:13.637696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad2375-f6a8-45e3-875d-99c5690afda8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:13.638131 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad2375-f6a8-45e3-875d-99c5690afda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:13.638614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad2375-f6a8-45e3-875d-99c5690afda8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:13.798091 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad2375-f6a8-45e3-875d-99c5690afda8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:13.800228 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad2375-f6a8-45e3-875d-99c5690afda8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:13.808542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c0b34-c63e-46fe-b2d1-d6de130a92c9 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:13.937189 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:13.948617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c0b34-c63e-46fe-b2d1-d6de130a92c9 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:13.995446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177e0f8-7f82-4507-a33e-d712dc0e5ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:13.996095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177e0f8-7f82-4507-a33e-d712dc0e5ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:13.996378 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177e0f8-7f82-4507-a33e-d712dc0e5ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:13.996656 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177e0f8-7f82-4507-a33e-d712dc0e5ec5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:14.019816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177e0f8-7f82-4507-a33e-d712dc0e5ec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-1177e0f8-7f82-4507-a33e-d712dc0e5ec5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:14.025357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:14.026296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:14.027109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:14.028200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 210/422] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:30:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:14.035547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177e0f8-7f82-4507-a33e-d712dc0e5ec5 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:14.035804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 213/423] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:14.050151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad2375-f6a8-45e3-875d-99c5690afda8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:14.109047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:14.199970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177e0f8-7f82-4507-a33e-d712dc0e5ec5 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:14.245014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbe466-04f3-4024-8a3f-d9557a0bbc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:14.247448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbe466-04f3-4024-8a3f-d9557a0bbc16 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:14.247448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbe466-04f3-4024-8a3f-d9557a0bbc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:14.247448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbe466-04f3-4024-8a3f-d9557a0bbc16 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:14.247448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbe466-04f3-4024-8a3f-d9557a0bbc16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:14.247448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbe466-04f3-4024-8a3f-d9557a0bbc16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:14.248567 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbe466-04f3-4024-8a3f-d9557a0bbc16 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:14.263494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 211/424] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:14.265049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0921752-00da-46a6-8831-a7cd34cfa036 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:14.265335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0921752-00da-46a6-8831-a7cd34cfa036 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:14.265611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0921752-00da-46a6-8831-a7cd34cfa036 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:14.267029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 212/425] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:14.273386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0921752-00da-46a6-8831-a7cd34cfa036 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:14.282363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad2375-f6a8-45e3-875d-99c5690afda8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:14.330449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d0e25-094e-48fb-9a8d-ce97fee483d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:14.330449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d0e25-094e-48fb-9a8d-ce97fee483d5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:14.330449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d0e25-094e-48fb-9a8d-ce97fee483d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:14.330449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d0e25-094e-48fb-9a8d-ce97fee483d5 None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:14.330866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d0e25-094e-48fb-9a8d-ce97fee483d5 None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:14.331784 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d0e25-094e-48fb-9a8d-ce97fee483d5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:14.333879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 214/426] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:14 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:14.335390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193bc26-848f-41b6-a65c-1e4b0a576d8f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:14.335672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193bc26-848f-41b6-a65c-1e4b0a576d8f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:14.335948 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193bc26-848f-41b6-a65c-1e4b0a576d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:14.361105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4193bc26-848f-41b6-a65c-1e4b0a576d8f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:14.362699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4193bc26-848f-41b6-a65c-1e4b0a576d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:14.385886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:14.400025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:14.406171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0921752-00da-46a6-8831-a7cd34cfa036 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:14.457871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd003f-64ea-4e8e-aba9-3c4c2f27ecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:14.459049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd003f-64ea-4e8e-aba9-3c4c2f27ecd6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:14.459681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd003f-64ea-4e8e-aba9-3c4c2f27ecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:14.460173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd003f-64ea-4e8e-aba9-3c4c2f27ecd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:14.478610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 215/427] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:14.479362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193bc26-848f-41b6-a65c-1e4b0a576d8f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:14.619711 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193bc26-848f-41b6-a65c-1e4b0a576d8f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:14.663052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29a82c-c90e-4f3e-8c80-c4af5627a84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:14.663971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29a82c-c90e-4f3e-8c80-c4af5627a84a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:14.664249 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29a82c-c90e-4f3e-8c80-c4af5627a84a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:14.664518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29a82c-c90e-4f3e-8c80-c4af5627a84a None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:14.665511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd003f-64ea-4e8e-aba9-3c4c2f27ecd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:14.665802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29a82c-c90e-4f3e-8c80-c4af5627a84a None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:14.667285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29a82c-c90e-4f3e-8c80-c4af5627a84a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:14.668603 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd003f-64ea-4e8e-aba9-3c4c2f27ecd6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:14.668883 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 216/428] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:14 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:14.678479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29a82c-c90e-4f3e-8c80-c4af5627a84a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:14.733678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:14.821144 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:14.830558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 213/429] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:14.832118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd003f-64ea-4e8e-aba9-3c4c2f27ecd6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:14.880059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29a82c-c90e-4f3e-8c80-c4af5627a84a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:14.930117 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb533b0-bf6d-49d1-9054-65485d8d2e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:14.931025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb533b0-bf6d-49d1-9054-65485d8d2e22 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:14.931306 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb533b0-bf6d-49d1-9054-65485d8d2e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:14.931578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb533b0-bf6d-49d1-9054-65485d8d2e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:14.932320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb533b0-bf6d-49d1-9054-65485d8d2e22 None admin] RBAC: Adding query filter params (group.id=7bb83bf8d47b4883ad7048ffdd699359, role.id=4d98a2b16f8f42bca3e5dac0374cdb54, scope.project.id=290b724bdca948a29018c8ecc35d1c8b) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:14.932625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb533b0-bf6d-49d1-9054-65485d8d2e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:14.935205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb533b0-bf6d-49d1-9054-65485d8d2e22 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:14.947095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 217/430] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Aug 29 23:30:14 2022] GET /identity/v3/role_assignments?group.id=7bb83bf8d47b4883ad7048ffdd699359&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 611 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:14.949807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb533b0-bf6d-49d1-9054-65485d8d2e22 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:15.017474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd003f-64ea-4e8e-aba9-3c4c2f27ecd6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:15.101311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf881c-1d2c-404b-bf4d-d2225605a65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:15.101715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf881c-1d2c-404b-bf4d-d2225605a65b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:15.101715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf881c-1d2c-404b-bf4d-d2225605a65b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:15.102109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf881c-1d2c-404b-bf4d-d2225605a65b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:15.133837 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf881c-1d2c-404b-bf4d-d2225605a65b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-2abf881c-1d2c-404b-bf4d-d2225605a65b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:15.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:15.140888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:15.143000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 214/431] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:15.148202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf881c-1d2c-404b-bf4d-d2225605a65b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:15.172086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:15.191035 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb533b0-bf6d-49d1-9054-65485d8d2e22 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:15.234542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:15.240402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576183f0-ffe5-45e7-9673-23fd61aa67c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:15.241382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576183f0-ffe5-45e7-9673-23fd61aa67c5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:15.241671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576183f0-ffe5-45e7-9673-23fd61aa67c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:15.241953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576183f0-ffe5-45e7-9673-23fd61aa67c5 None admin] PATH_INFO: `/v3/projects/290b724bdca948a29018c8ecc35d1c8b` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:15.243375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576183f0-ffe5-45e7-9673-23fd61aa67c5 None admin] RBAC: Authorizing `identity:get_project(project_id=290b724bdca948a29018c8ecc35d1c8b)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:15.250676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576183f0-ffe5-45e7-9673-23fd61aa67c5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:15.250676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 218/432] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:14 2022] GET /identity/v3/projects/290b724bdca948a29018c8ecc35d1c8b => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:15.253216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576183f0-ffe5-45e7-9673-23fd61aa67c5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:15.304177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf881c-1d2c-404b-bf4d-d2225605a65b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:15.347881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c63dc4-741e-4c76-a052-372d7ce14d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:15.349684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c63dc4-741e-4c76-a052-372d7ce14d58 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:15.351523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c63dc4-741e-4c76-a052-372d7ce14d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:15.351523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c63dc4-741e-4c76-a052-372d7ce14d58 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:15.351523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c63dc4-741e-4c76-a052-372d7ce14d58 None admin] RBAC: Adding query filter params (name=member) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:15.351789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c63dc4-741e-4c76-a052-372d7ce14d58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:15.353119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c63dc4-741e-4c76-a052-372d7ce14d58 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:15.366976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 215/433] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:30:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:15.368294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753925e-370f-4c12-86a9-d57d8368cdc2 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:15.368583 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753925e-370f-4c12-86a9-d57d8368cdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:15.368860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753925e-370f-4c12-86a9-d57d8368cdc2 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:15.371353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 216/434] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:15.374158 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753925e-370f-4c12-86a9-d57d8368cdc2 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:15.403945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576183f0-ffe5-45e7-9673-23fd61aa67c5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:15.456746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e888cd5-955a-443b-98c1-1ba7fa35d1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:15.457402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e888cd5-955a-443b-98c1-1ba7fa35d1fb None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:15.457686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e888cd5-955a-443b-98c1-1ba7fa35d1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:15.457963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e888cd5-955a-443b-98c1-1ba7fa35d1fb None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:15.460799 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e888cd5-955a-443b-98c1-1ba7fa35d1fb None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:15.461370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e888cd5-955a-443b-98c1-1ba7fa35d1fb None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:15.463513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 219/435] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:15 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:15.464753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb322a8-a67a-44af-b96e-5e31c48164a2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:15.465034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb322a8-a67a-44af-b96e-5e31c48164a2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:15.465309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb322a8-a67a-44af-b96e-5e31c48164a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:15.485496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5eb322a8-a67a-44af-b96e-5e31c48164a2 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:15.486764 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5eb322a8-a67a-44af-b96e-5e31c48164a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:15.516615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:15.532588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753925e-370f-4c12-86a9-d57d8368cdc2 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:15.560427 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:15.575790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cff9b-d079-4643-8c0c-b488a679aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:15.576403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cff9b-d079-4643-8c0c-b488a679aac6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:15.576681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cff9b-d079-4643-8c0c-b488a679aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:15.576951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cff9b-d079-4643-8c0c-b488a679aac6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:15.579242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 220/436] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:15.590669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a28ef2-4eb6-45e9-bada-2d180b2eb0bd None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:15.590669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a28ef2-4eb6-45e9-bada-2d180b2eb0bd None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:15.590669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a28ef2-4eb6-45e9-bada-2d180b2eb0bd None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:15.590669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 221/437] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:15.590669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a28ef2-4eb6-45e9-bada-2d180b2eb0bd None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:15.662019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cff9b-d079-4643-8c0c-b488a679aac6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:15.663626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cff9b-d079-4643-8c0c-b488a679aac6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:15.728630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a28ef2-4eb6-45e9-bada-2d180b2eb0bd None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:15.781428 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b3c3a-64c0-4a8f-b959-b061554a7552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:15.782088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 217/438] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 23:30:15.782578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b3c3a-64c0-4a8f-b959-b061554a7552 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:15.783010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b3c3a-64c0-4a8f-b959-b061554a7552 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:15.783377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e581-d4b5-4c85-b96c-cc6742598248 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:15.783623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b3c3a-64c0-4a8f-b959-b061554a7552 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:15.783719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e581-d4b5-4c85-b96c-cc6742598248 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:15.783916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e581-d4b5-4c85-b96c-cc6742598248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:15.784784 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b3c3a-64c0-4a8f-b959-b061554a7552 None admin] RBAC: Adding query filter params (group.id=7bb83bf8d47b4883ad7048ffdd699359, role.id=a38face2eeae40c28edda83a4d46614f, scope.project.id=290b724bdca948a29018c8ecc35d1c8b) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:15.785255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b3c3a-64c0-4a8f-b959-b061554a7552 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:15.787679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b3c3a-64c0-4a8f-b959-b061554a7552 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:15.795725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 222/439] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Aug 29 23:30:15 2022] GET /identity/v3/role_assignments?group.id=7bb83bf8d47b4883ad7048ffdd699359&scope.project.id=290b724bdca948a29018c8ecc35d1c8b&role.id=a38face2eeae40c28edda83a4d46614f => generated 611 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:15.796847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b3c3a-64c0-4a8f-b959-b061554a7552 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:15.804083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-e068e581-d4b5-4c85-b96c-cc6742598248 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:15.805321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e068e581-d4b5-4c85-b96c-cc6742598248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:15.824189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:15.847218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:15.893237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 218/440] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:15.897931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e581-d4b5-4c85-b96c-cc6742598248 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:15.989120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b3c3a-64c0-4a8f-b959-b061554a7552 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:16.033014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca7f38-d03d-40f3-8f00-6f73841097e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:16.034035 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca7f38-d03d-40f3-8f00-6f73841097e5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:16.034685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca7f38-d03d-40f3-8f00-6f73841097e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:16.035374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca7f38-d03d-40f3-8f00-6f73841097e5 None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:16.037728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca7f38-d03d-40f3-8f00-6f73841097e5 None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:16.040284 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca7f38-d03d-40f3-8f00-6f73841097e5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:16.042252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 223/441] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:15 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:16.042685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e581-d4b5-4c85-b96c-cc6742598248 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:16.048568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca7f38-d03d-40f3-8f00-6f73841097e5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:16.104042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dc2d9-0de3-48d3-9efe-ab5a5f56a138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:16.104042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc2d9-0de3-48d3-9efe-ab5a5f56a138 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:16.104042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc2d9-0de3-48d3-9efe-ab5a5f56a138 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:16.104392 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc2d9-0de3-48d3-9efe-ab5a5f56a138 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:16.123071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734dc2d9-0de3-48d3-9efe-ab5a5f56a138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-734dc2d9-0de3-48d3-9efe-ab5a5f56a138 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:16.128462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:16.129374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:16.130780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 219/442] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:30:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:16.136904 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dc2d9-0de3-48d3-9efe-ab5a5f56a138 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:16.160763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:16.188314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:16.221516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca7f38-d03d-40f3-8f00-6f73841097e5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:16.269356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21c452-5648-4d38-aa7e-8cc970f93f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:16.270278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21c452-5648-4d38-aa7e-8cc970f93f45 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:16.270614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21c452-5648-4d38-aa7e-8cc970f93f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:16.270889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21c452-5648-4d38-aa7e-8cc970f93f45 None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:16.272089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21c452-5648-4d38-aa7e-8cc970f93f45 None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:16.273507 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21c452-5648-4d38-aa7e-8cc970f93f45 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:16.275150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 224/443] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:16 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:16.275940 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21c452-5648-4d38-aa7e-8cc970f93f45 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:16.295710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dc2d9-0de3-48d3-9efe-ab5a5f56a138 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:16.337892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea271783-24b7-4eae-8396-0110ff7b3500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:16.338527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea271783-24b7-4eae-8396-0110ff7b3500 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:16.338801 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea271783-24b7-4eae-8396-0110ff7b3500 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:16.339072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea271783-24b7-4eae-8396-0110ff7b3500 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:16.339734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea271783-24b7-4eae-8396-0110ff7b3500 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:16.340028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea271783-24b7-4eae-8396-0110ff7b3500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:16.341406 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea271783-24b7-4eae-8396-0110ff7b3500 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:16.353460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 220/444] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:16.358569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea271783-24b7-4eae-8396-0110ff7b3500 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:16.401919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21c452-5648-4d38-aa7e-8cc970f93f45 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:16.438094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:16.439078 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e580a27-8d73-4c20-a8c0-0895732d1b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:16.441693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580a27-8d73-4c20-a8c0-0895732d1b5f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:16.441693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580a27-8d73-4c20-a8c0-0895732d1b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:16.441693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580a27-8d73-4c20-a8c0-0895732d1b5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:16.451383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e580a27-8d73-4c20-a8c0-0895732d1b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1e580a27-8d73-4c20-a8c0-0895732d1b5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:16.456727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:16.457732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 225/445] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:30:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:16.465305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e580a27-8d73-4c20-a8c0-0895732d1b5f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:16.472892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea271783-24b7-4eae-8396-0110ff7b3500 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:16.475595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:16.511793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2d8ac-9712-49ba-a850-01037d61e8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:16.512396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2d8ac-9712-49ba-a850-01037d61e8bf None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:16.512672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2d8ac-9712-49ba-a850-01037d61e8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:16.512938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2d8ac-9712-49ba-a850-01037d61e8bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:16.513831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2d8ac-9712-49ba-a850-01037d61e8bf None admin] RBAC: Adding query filter params (group.id=7bb83bf8d47b4883ad7048ffdd699359, role.id=4d98a2b16f8f42bca3e5dac0374cdb54, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:16.514131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2d8ac-9712-49ba-a850-01037d61e8bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:16.517364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2d8ac-9712-49ba-a850-01037d61e8bf None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:16.523340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 221/446] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Aug 29 23:30:16 2022] GET /identity/v3/role_assignments?group.id=7bb83bf8d47b4883ad7048ffdd699359&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=4d98a2b16f8f42bca3e5dac0374cdb54 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:16.524167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2d8ac-9712-49ba-a850-01037d61e8bf None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:16.586186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e580a27-8d73-4c20-a8c0-0895732d1b5f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:16.627709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec68f3e-64a6-4b1d-a453-ae8966e5a094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:16.628249 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec68f3e-64a6-4b1d-a453-ae8966e5a094 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:16.628639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec68f3e-64a6-4b1d-a453-ae8966e5a094 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:16.628914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec68f3e-64a6-4b1d-a453-ae8966e5a094 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:16.656346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2d8ac-9712-49ba-a850-01037d61e8bf None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:16.705987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3385b1e-ca21-42da-804c-c5fa930a2d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:16.708955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3385b1e-ca21-42da-804c-c5fa930a2d31 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:16.708955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3385b1e-ca21-42da-804c-c5fa930a2d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:16.708955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3385b1e-ca21-42da-804c-c5fa930a2d31 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:16.708955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3385b1e-ca21-42da-804c-c5fa930a2d31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:16.708955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3385b1e-ca21-42da-804c-c5fa930a2d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:16.710083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3385b1e-ca21-42da-804c-c5fa930a2d31 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:16.712653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec68f3e-64a6-4b1d-a453-ae8966e5a094 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:16.713209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:16.714146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec68f3e-64a6-4b1d-a453-ae8966e5a094 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:16.721064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 222/447] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:16.729038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3385b1e-ca21-42da-804c-c5fa930a2d31 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:16.734852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:16.847805 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3385b1e-ca21-42da-804c-c5fa930a2d31 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:16.854590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 226/448] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:16.859471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec68f3e-64a6-4b1d-a453-ae8966e5a094 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:16.887551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d1cd2-2fd1-4d61-9997-ae39576297dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:16.889234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d1cd2-2fd1-4d61-9997-ae39576297dd None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:16.889581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d1cd2-2fd1-4d61-9997-ae39576297dd None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:16.889950 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d1cd2-2fd1-4d61-9997-ae39576297dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:16.960196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:16.964050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d1cd2-2fd1-4d61-9997-ae39576297dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:16.965494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d1cd2-2fd1-4d61-9997-ae39576297dd None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:16.972741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec68f3e-64a6-4b1d-a453-ae8966e5a094 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:16.986159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:17.007819 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa338cbe-f969-454c-a2d4-efe870ed6871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:17.008487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa338cbe-f969-454c-a2d4-efe870ed6871 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:17.008725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa338cbe-f969-454c-a2d4-efe870ed6871 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:17.008999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa338cbe-f969-454c-a2d4-efe870ed6871 None admin] PATH_INFO: `/v3/projects/291cfd89a4d34a9da9d99e5ca0c9d4dd` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:17.010145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa338cbe-f969-454c-a2d4-efe870ed6871 None admin] RBAC: Authorizing `identity:get_project(project_id=291cfd89a4d34a9da9d99e5ca0c9d4dd)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:17.011649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa338cbe-f969-454c-a2d4-efe870ed6871 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:17.013263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 227/449] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:16 2022] GET /identity/v3/projects/291cfd89a4d34a9da9d99e5ca0c9d4dd => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:17.018802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa338cbe-f969-454c-a2d4-efe870ed6871 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:17.068817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 223/450] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 23:30:17.074526 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d1cd2-2fd1-4d61-9997-ae39576297dd None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:17.126349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa338cbe-f969-454c-a2d4-efe870ed6871 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:17.162574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70febe66-a2e6-439a-8d19-447d8e281ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:17.163132 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70febe66-a2e6-439a-8d19-447d8e281ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:17.163461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70febe66-a2e6-439a-8d19-447d8e281ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:17.163733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70febe66-a2e6-439a-8d19-447d8e281ea9 None admin] PATH_INFO: `/v3/groups/ce6e0c80918444debacfb0d6b84bf680` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:17.164865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70febe66-a2e6-439a-8d19-447d8e281ea9 None admin] RBAC: Authorizing `identity:get_group(group_id=ce6e0c80918444debacfb0d6b84bf680)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:17.166294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70febe66-a2e6-439a-8d19-447d8e281ea9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:17.167875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 228/451] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:17 2022] GET /identity/v3/groups/ce6e0c80918444debacfb0d6b84bf680 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:17.173585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70febe66-a2e6-439a-8d19-447d8e281ea9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:17.175863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:17.179789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d1cd2-2fd1-4d61-9997-ae39576297dd None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:17.216515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7c291-6da7-4fb1-9b94-948bbf59b61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:17.218755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7c291-6da7-4fb1-9b94-948bbf59b61e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:17.218755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7c291-6da7-4fb1-9b94-948bbf59b61e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:17.218755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7c291-6da7-4fb1-9b94-948bbf59b61e None admin] PATH_INFO: `/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:17.219009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7c291-6da7-4fb1-9b94-948bbf59b61e None admin] RBAC: Authorizing `identity:get_project(project_id=45aa2f6db17f4df1b7f892a95d31b8fc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:17.220226 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7c291-6da7-4fb1-9b94-948bbf59b61e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:17.221827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 224/452] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:30:17 2022] GET /identity/v3/projects/45aa2f6db17f4df1b7f892a95d31b8fc => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:17.222554 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:17.228224 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7c291-6da7-4fb1-9b94-948bbf59b61e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:17.276622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70febe66-a2e6-439a-8d19-447d8e281ea9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:17.311574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0729b1-1e7c-4f75-a94c-e3251eaf139a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:17.312170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0729b1-1e7c-4f75-a94c-e3251eaf139a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:17.312448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0729b1-1e7c-4f75-a94c-e3251eaf139a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:17.312752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0729b1-1e7c-4f75-a94c-e3251eaf139a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:17.313431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0729b1-1e7c-4f75-a94c-e3251eaf139a None admin] RBAC: Adding query filter params (group.id=ce6e0c80918444debacfb0d6b84bf680, role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=291cfd89a4d34a9da9d99e5ca0c9d4dd) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:17.313738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0729b1-1e7c-4f75-a94c-e3251eaf139a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:17.315197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0729b1-1e7c-4f75-a94c-e3251eaf139a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:17.322012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 229/453] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Aug 29 23:30:17 2022] GET /identity/v3/role_assignments?group.id=ce6e0c80918444debacfb0d6b84bf680&scope.project.id=291cfd89a4d34a9da9d99e5ca0c9d4dd&role.id=d86dba8efe20423abdde06031690beb8 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:17.333733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7c291-6da7-4fb1-9b94-948bbf59b61e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:17.374111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd3f9b-59ae-46fd-a684-25a9b939a768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:17.374799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd3f9b-59ae-46fd-a684-25a9b939a768 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:17.375079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd3f9b-59ae-46fd-a684-25a9b939a768 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:17.375352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd3f9b-59ae-46fd-a684-25a9b939a768 None admin] PATH_INFO: `/v3/groups/7bb83bf8d47b4883ad7048ffdd699359` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:17.376559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd3f9b-59ae-46fd-a684-25a9b939a768 None admin] RBAC: Authorizing `identity:get_group(group_id=7bb83bf8d47b4883ad7048ffdd699359)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:17.378087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd3f9b-59ae-46fd-a684-25a9b939a768 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:17.380487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 225/454] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Aug 29 23:30:17 2022] GET /identity/v3/groups/7bb83bf8d47b4883ad7048ffdd699359 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:17.386889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0729b1-1e7c-4f75-a94c-e3251eaf139a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:17.499540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0729b1-1e7c-4f75-a94c-e3251eaf139a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:17.514177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:17.537429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679461c-ef43-4172-9d20-9a1d4d4cb783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:17.538103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679461c-ef43-4172-9d20-9a1d4d4cb783 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:17.538378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679461c-ef43-4172-9d20-9a1d4d4cb783 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:17.538688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679461c-ef43-4172-9d20-9a1d4d4cb783 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:17.539411 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0679461c-ef43-4172-9d20-9a1d4d4cb783 None admin] RBAC: Adding query filter params (group.id=7bb83bf8d47b4883ad7048ffdd699359, role.id=a38face2eeae40c28edda83a4d46614f, scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:17.539719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0679461c-ef43-4172-9d20-9a1d4d4cb783 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:17.541153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0679461c-ef43-4172-9d20-9a1d4d4cb783 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:17.547481 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 230/455] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Aug 29 23:30:17 2022] GET /identity/v3/role_assignments?group.id=7bb83bf8d47b4883ad7048ffdd699359&scope.project.id=45aa2f6db17f4df1b7f892a95d31b8fc&role.id=a38face2eeae40c28edda83a4d46614f => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:20.047132 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c74d6-6b27-4c51-a3ba-fd2b2c5f349e None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:20.047679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c74d6-6b27-4c51-a3ba-fd2b2c5f349e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:20.047679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c74d6-6b27-4c51-a3ba-fd2b2c5f349e None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:20.048996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 226/456] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:20.055335 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6a0ab-1c82-453f-be89-3105ebd9d414 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:20.055788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6a0ab-1c82-453f-be89-3105ebd9d414 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:20.055883 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6a0ab-1c82-453f-be89-3105ebd9d414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:20.079203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-32b6a0ab-1c82-453f-be89-3105ebd9d414 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:20.080751 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-32b6a0ab-1c82-453f-be89-3105ebd9d414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:20.172589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 231/457] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:20.593306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7074a-f9d8-47f1-8bc6-98d986c5c1d1 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:20.593595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7074a-f9d8-47f1-8bc6-98d986c5c1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:20.593855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7074a-f9d8-47f1-8bc6-98d986c5c1d1 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:20.595556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 227/458] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:20.601983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ef298-3b35-4e54-9127-fa46933e70cd None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:20.602188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ef298-3b35-4e54-9127-fa46933e70cd None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:20.602478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ef298-3b35-4e54-9127-fa46933e70cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:20.628115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56953cda-1016-400b-a109-17f36d1bd925 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:20.628535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56953cda-1016-400b-a109-17f36d1bd925 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:20.628923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56953cda-1016-400b-a109-17f36d1bd925 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:20.635894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 228/459] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:20.640022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-811ef298-3b35-4e54-9127-fa46933e70cd None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:20.641296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-811ef298-3b35-4e54-9127-fa46933e70cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:20.649754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1de5d-59e4-4612-8aff-2f49cc7af039 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:20.650045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1de5d-59e4-4612-8aff-2f49cc7af039 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:20.650321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1de5d-59e4-4612-8aff-2f49cc7af039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:20.670282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-76a1de5d-59e4-4612-8aff-2f49cc7af039 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:20.671539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-76a1de5d-59e4-4612-8aff-2f49cc7af039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:20.719293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:20.732200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 232/460] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:20.741384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ef298-3b35-4e54-9127-fa46933e70cd None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:20.784430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:20.837253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 229/461] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:20.952638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ef298-3b35-4e54-9127-fa46933e70cd None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:21.019270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c21adb-fc79-4aaa-8e9f-4a573b395341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:21.020741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c21adb-fc79-4aaa-8e9f-4a573b395341 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:21.021708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c21adb-fc79-4aaa-8e9f-4a573b395341 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:21.021990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c21adb-fc79-4aaa-8e9f-4a573b395341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:21.033636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c21adb-fc79-4aaa-8e9f-4a573b395341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:21.043449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c21adb-fc79-4aaa-8e9f-4a573b395341 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-65c21adb-fc79-4aaa-8e9f-4a573b395341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:21.066800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:21.069204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:21.070047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:21.070047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:21.070047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:21.074993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 233/462] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:21.083498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1de5d-59e4-4612-8aff-2f49cc7af039 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:21.124615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c21adb-fc79-4aaa-8e9f-4a573b395341 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:21.260961 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1de5d-59e4-4612-8aff-2f49cc7af039 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:21.304722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:21.315312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36f243-3ad3-44a4-9b2b-0a4da5c0ae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:21.316713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36f243-3ad3-44a4-9b2b-0a4da5c0ae2e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:21.318464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36f243-3ad3-44a4-9b2b-0a4da5c0ae2e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:21.326859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36f243-3ad3-44a4-9b2b-0a4da5c0ae2e None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:21.328566 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36f243-3ad3-44a4-9b2b-0a4da5c0ae2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:21.329356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36f243-3ad3-44a4-9b2b-0a4da5c0ae2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:21.330058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c21adb-fc79-4aaa-8e9f-4a573b395341 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:21.332402 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36f243-3ad3-44a4-9b2b-0a4da5c0ae2e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:21.366860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 230/463] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:21.368683 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36f243-3ad3-44a4-9b2b-0a4da5c0ae2e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:21.370847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c44eb-70a9-4875-a58d-dc61be1dfec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:21.371514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c44eb-70a9-4875-a58d-dc61be1dfec0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:21.371754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c44eb-70a9-4875-a58d-dc61be1dfec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:21.372020 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c44eb-70a9-4875-a58d-dc61be1dfec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:21.390170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c44eb-70a9-4875-a58d-dc61be1dfec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:21.393879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c44eb-70a9-4875-a58d-dc61be1dfec0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e62c44eb-70a9-4875-a58d-dc61be1dfec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:21.413210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:21.415005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:21.416685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:21.416685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:21.416685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:21.418592 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 234/464] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:21.422554 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c44eb-70a9-4875-a58d-dc61be1dfec0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:21.425842 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:21.526056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36f243-3ad3-44a4-9b2b-0a4da5c0ae2e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:21.564423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26bab2-4115-4f9e-b583-ae438258dcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:21.565065 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26bab2-4115-4f9e-b583-ae438258dcea None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:21.565348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26bab2-4115-4f9e-b583-ae438258dcea None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:21.565686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26bab2-4115-4f9e-b583-ae438258dcea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:21.569852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c44eb-70a9-4875-a58d-dc61be1dfec0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:21.577468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26bab2-4115-4f9e-b583-ae438258dcea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:21.578962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26bab2-4115-4f9e-b583-ae438258dcea None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-cd26bab2-4115-4f9e-b583-ae438258dcea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:21.596208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:21.597166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:21.598043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:21.598043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:21.598043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:21.598043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 231/465] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:21.599287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26bab2-4115-4f9e-b583-ae438258dcea None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:21.616518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4749c-b6d6-4332-967e-3f46ab9ba338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:21.617131 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4749c-b6d6-4332-967e-3f46ab9ba338 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:21.617446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4749c-b6d6-4332-967e-3f46ab9ba338 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:21.617738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4749c-b6d6-4332-967e-3f46ab9ba338 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:21.618448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4749c-b6d6-4332-967e-3f46ab9ba338 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:21.620001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4749c-b6d6-4332-967e-3f46ab9ba338 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:21.632020 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:21.683687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:21.705638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26bab2-4115-4f9e-b583-ae438258dcea None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:21.709994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 235/466] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:30:21 2022] POST /identity/v3/users => generated 246 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:30:21.715084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d322472-4c95-463f-a17e-788f951b992a None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:21.715084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d322472-4c95-463f-a17e-788f951b992a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:21.715084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d322472-4c95-463f-a17e-788f951b992a None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:21.715084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 236/467] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:21.718128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d322472-4c95-463f-a17e-788f951b992a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:21.760721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce7f61-e2e7-4b11-ad3c-d87dff08c7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:21.762111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7f61-e2e7-4b11-ad3c-d87dff08c7ca None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:21.762619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7f61-e2e7-4b11-ad3c-d87dff08c7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:21.763245 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7f61-e2e7-4b11-ad3c-d87dff08c7ca None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:21.764635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce7f61-e2e7-4b11-ad3c-d87dff08c7ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:21.765510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce7f61-e2e7-4b11-ad3c-d87dff08c7ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:21.767416 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce7f61-e2e7-4b11-ad3c-d87dff08c7ca None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:21.790090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 232/468] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:21.791192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede7956-5148-418d-b716-a76edc5eb8c3 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:21.791483 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede7956-5148-418d-b716-a76edc5eb8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:21.791758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede7956-5148-418d-b716-a76edc5eb8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:21.815991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-dede7956-5148-418d-b716-a76edc5eb8c3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:21.818397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-dede7956-5148-418d-b716-a76edc5eb8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:21.884839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d322472-4c95-463f-a17e-788f951b992a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:21.916069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 233/469] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:21.917145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede7956-5148-418d-b716-a76edc5eb8c3 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:21.924597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90decf36-1d90-4331-b797-8a9ccfcdee07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:21.925507 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90decf36-1d90-4331-b797-8a9ccfcdee07 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:21.926034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90decf36-1d90-4331-b797-8a9ccfcdee07 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:21.926691 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90decf36-1d90-4331-b797-8a9ccfcdee07 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:21.927855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90decf36-1d90-4331-b797-8a9ccfcdee07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:21.928432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90decf36-1d90-4331-b797-8a9ccfcdee07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:21.930136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90decf36-1d90-4331-b797-8a9ccfcdee07 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:21.954877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 237/470] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:21.961852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90decf36-1d90-4331-b797-8a9ccfcdee07 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:22.016119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:22.084876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede7956-5148-418d-b716-a76edc5eb8c3 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:22.088162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:22.157298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209fe57-8fb8-4da1-bfd8-5d4c2d307f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:22.158177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209fe57-8fb8-4da1-bfd8-5d4c2d307f39 None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:22.158479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209fe57-8fb8-4da1-bfd8-5d4c2d307f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:22.158756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209fe57-8fb8-4da1-bfd8-5d4c2d307f39 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:22.161112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209fe57-8fb8-4da1-bfd8-5d4c2d307f39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:22.162524 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209fe57-8fb8-4da1-bfd8-5d4c2d307f39 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:22.170455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90decf36-1d90-4331-b797-8a9ccfcdee07 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:22.208370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4667137-e872-404f-8a40-e5397db845eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:22.208988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667137-e872-404f-8a40-e5397db845eb None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:22.209268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667137-e872-404f-8a40-e5397db845eb None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:22.209541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667137-e872-404f-8a40-e5397db845eb None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:22.210194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4667137-e872-404f-8a40-e5397db845eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:22.211686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4667137-e872-404f-8a40-e5397db845eb None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:22.326996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 234/471] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:30:21 2022] POST /identity/v3/users => generated 248 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:30:22.326996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57921435-07ba-4cf4-a2c2-9be2aef7a95e None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:22.326996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57921435-07ba-4cf4-a2c2-9be2aef7a95e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:22.326996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57921435-07ba-4cf4-a2c2-9be2aef7a95e None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:22.326996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 235/472] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:22.326996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57921435-07ba-4cf4-a2c2-9be2aef7a95e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:22.361788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 238/473] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:30:21 2022] POST /identity/v3/users => generated 248 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:30:22.363567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48b8a8-cfa2-46c7-ac45-0f185ab9677f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:22.364325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48b8a8-cfa2-46c7-ac45-0f185ab9677f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:22.365544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48b8a8-cfa2-46c7-ac45-0f185ab9677f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:22.392137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9b48b8a8-cfa2-46c7-ac45-0f185ab9677f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:22.393966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9b48b8a8-cfa2-46c7-ac45-0f185ab9677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:22.544992 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57921435-07ba-4cf4-a2c2-9be2aef7a95e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:22.546850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 239/474] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:22.578754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:22.594976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341109b4-8a38-4ab7-930b-bae29a9dc68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:22.595601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341109b4-8a38-4ab7-930b-bae29a9dc68f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:22.601180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341109b4-8a38-4ab7-930b-bae29a9dc68f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:22.601180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341109b4-8a38-4ab7-930b-bae29a9dc68f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:22.610364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341109b4-8a38-4ab7-930b-bae29a9dc68f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:22.612179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341109b4-8a38-4ab7-930b-bae29a9dc68f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-341109b4-8a38-4ab7-930b-bae29a9dc68f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:22.635940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:22.646177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:22.647115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:22.647115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:22.647115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:22.647115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 236/475] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:22.647339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48b8a8-cfa2-46c7-ac45-0f185ab9677f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:22.705451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:22.776975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48b8a8-cfa2-46c7-ac45-0f185ab9677f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:22.814765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cacbe6-a809-4def-8f16-67d6fa8bbb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:22.815380 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cacbe6-a809-4def-8f16-67d6fa8bbb70 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:22.815658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cacbe6-a809-4def-8f16-67d6fa8bbb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:22.815930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cacbe6-a809-4def-8f16-67d6fa8bbb70 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:22.816577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cacbe6-a809-4def-8f16-67d6fa8bbb70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:22.817418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cacbe6-a809-4def-8f16-67d6fa8bbb70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:22.818847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cacbe6-a809-4def-8f16-67d6fa8bbb70 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:22.831404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 240/476] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:22.832858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341109b4-8a38-4ab7-930b-bae29a9dc68f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:22.837257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cacbe6-a809-4def-8f16-67d6fa8bbb70 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:22.947254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cacbe6-a809-4def-8f16-67d6fa8bbb70 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:22.949403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:22.985649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a83be-efa6-4de7-8503-8738f0840e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:22.986596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a83be-efa6-4de7-8503-8738f0840e5e None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:22.987076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a83be-efa6-4de7-8503-8738f0840e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:22.987522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a83be-efa6-4de7-8503-8738f0840e5e None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:22.988359 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a83be-efa6-4de7-8503-8738f0840e5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:22.990983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a83be-efa6-4de7-8503-8738f0840e5e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:23.061592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341109b4-8a38-4ab7-930b-bae29a9dc68f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:23.088980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 241/477] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:30:22 2022] POST /identity/v3/users => generated 249 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:30:23.111079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee3ea-782a-4afb-822a-a9d4c36b2a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:23.112410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee3ea-782a-4afb-822a-a9d4c36b2a63 None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:23.112857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee3ea-782a-4afb-822a-a9d4c36b2a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:23.113289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee3ea-782a-4afb-822a-a9d4c36b2a63 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:23.114816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edee3ea-782a-4afb-822a-a9d4c36b2a63 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:23.116489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edee3ea-782a-4afb-822a-a9d4c36b2a63 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:23.167462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.sql.core [None req-7edee3ea-782a-4afb-822a-a9d4c36b2a63 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 23:30:23.167462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 23:30:23.167462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [parameters: {'id': 'c5e968616ca0491ab8dfbb66a2f101ab', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 23:30:23.167462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81853) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-7edee3ea-782a-4afb-822a-a9d4c36b2a63 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application conn.query(q) Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application result.read() Aug 29 23:30:23.193190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return ref.to_dict() Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 23:30:23.194141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application next(self.gen) Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application yield resource Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application next(self.gen) Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise self.value Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application session.commit() Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 23:30:23.195116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self.session.flush() Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application compat.raise_( Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 23:30:23.195989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 23:30:23.196895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application conn.query(q) Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application result.read() Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 23:30:23.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [parameters: {'id': 'c5e968616ca0491ab8dfbb66a2f101ab', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:23.201844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 23:30:23.202734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:23.205437 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 237/478] 10.222.0.58 () {60 vars in 1131 bytes} [Mon Aug 29 23:30:22 2022] POST /identity/v3/roles => generated 144 bytes in 373 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:23.213906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a83be-efa6-4de7-8503-8738f0840e5e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:23.386365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a83be-efa6-4de7-8503-8738f0840e5e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:23.422363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd02d2b-28dd-47e4-8618-c644ff620d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:23.423028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02d2b-28dd-47e4-8618-c644ff620d73 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:23.423309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02d2b-28dd-47e4-8618-c644ff620d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:23.423642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02d2b-28dd-47e4-8618-c644ff620d73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:23.436159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd02d2b-28dd-47e4-8618-c644ff620d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-abd02d2b-28dd-47e4-8618-c644ff620d73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:23.442967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:23.452059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:23.453449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 242/479] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:30:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:23.454106 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee3ea-782a-4afb-822a-a9d4c36b2a63 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:23.511638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:23.582666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee3ea-782a-4afb-822a-a9d4c36b2a63 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:23.621471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441ddaa-9968-4f36-adae-8b7b61bc7085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:23.622513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441ddaa-9968-4f36-adae-8b7b61bc7085 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:23.622988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441ddaa-9968-4f36-adae-8b7b61bc7085 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:23.623452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441ddaa-9968-4f36-adae-8b7b61bc7085 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:23.624405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441ddaa-9968-4f36-adae-8b7b61bc7085 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:23.624951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441ddaa-9968-4f36-adae-8b7b61bc7085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:23.626639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441ddaa-9968-4f36-adae-8b7b61bc7085 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:23.636503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 238/480] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:24.115763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db066-d960-46a1-8387-afac3cd8259a None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:24.116270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db066-d960-46a1-8387-afac3cd8259a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:24.116345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db066-d960-46a1-8387-afac3cd8259a None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:24.117639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 243/481] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:24.123043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc363e-6858-4213-a14c-199a6ed733dd None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:24.123466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc363e-6858-4213-a14c-199a6ed733dd None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:24.123639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc363e-6858-4213-a14c-199a6ed733dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:24.149924 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-e3cc363e-6858-4213-a14c-199a6ed733dd None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:24.151141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e3cc363e-6858-4213-a14c-199a6ed733dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:24.317074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 239/482] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:24.758613 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd127a-4fd2-4bde-8256-d77da73bc383 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:24.758992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd127a-4fd2-4bde-8256-d77da73bc383 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:24.759181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd127a-4fd2-4bde-8256-d77da73bc383 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:24.760747 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 244/483] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:24.766146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c62316-67be-4395-b35d-d226cf4a7f8a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:24.766452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c62316-67be-4395-b35d-d226cf4a7f8a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:24.766745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c62316-67be-4395-b35d-d226cf4a7f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:24.792503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-49c62316-67be-4395-b35d-d226cf4a7f8a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:24.793724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-49c62316-67be-4395-b35d-d226cf4a7f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:24.820943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:24.865837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa324173-d034-46e2-b3e4-df2f4ba740a3 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:24.866062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa324173-d034-46e2-b3e4-df2f4ba740a3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:24.866299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa324173-d034-46e2-b3e4-df2f4ba740a3 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:24.872084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 245/484] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:24.877200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3219-4b27-4373-9504-40f37c95057d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:24.877480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3219-4b27-4373-9504-40f37c95057d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:24.877838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3219-4b27-4373-9504-40f37c95057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:24.891233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 240/485] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:24.902865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-68df3219-4b27-4373-9504-40f37c95057d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:24.904184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-68df3219-4b27-4373-9504-40f37c95057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:24.913109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c62316-67be-4395-b35d-d226cf4a7f8a None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:24.948877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:25.075575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 246/486] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:30:25.106126 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c62316-67be-4395-b35d-d226cf4a7f8a None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:25.142444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4297877-8e60-4b4e-9694-3c2112f8d7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:25.143223 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4297877-8e60-4b4e-9694-3c2112f8d7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:25.143595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4297877-8e60-4b4e-9694-3c2112f8d7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:25.144007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4297877-8e60-4b4e-9694-3c2112f8d7b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:25.156525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4297877-8e60-4b4e-9694-3c2112f8d7b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-d4297877-8e60-4b4e-9694-3c2112f8d7b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:25.162615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:25.163687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:25.164675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:25.166355 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 241/487] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:25.172268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df3219-4b27-4373-9504-40f37c95057d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:25.185132 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4297877-8e60-4b4e-9694-3c2112f8d7b1 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:25.230259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:25.391298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df3219-4b27-4373-9504-40f37c95057d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:25.404501 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4297877-8e60-4b4e-9694-3c2112f8d7b1 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:25.440613 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91ab2d-8467-4bf7-9d49-031b41996255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:25.441227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91ab2d-8467-4bf7-9d49-031b41996255 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:25.441574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91ab2d-8467-4bf7-9d49-031b41996255 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:25.441850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91ab2d-8467-4bf7-9d49-031b41996255 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:25.442701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91ab2d-8467-4bf7-9d49-031b41996255 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:25.443018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91ab2d-8467-4bf7-9d49-031b41996255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:25.451459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91ab2d-8467-4bf7-9d49-031b41996255 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:25.451582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6332e8-7f15-4ce4-a798-677414c61a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:25.451582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6332e8-7f15-4ce4-a798-677414c61a88 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:25.451582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6332e8-7f15-4ce4-a798-677414c61a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:25.451582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6332e8-7f15-4ce4-a798-677414c61a88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:25.453791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 247/488] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:25.455027 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91ab2d-8467-4bf7-9d49-031b41996255 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:25.463938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6332e8-7f15-4ce4-a798-677414c61a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-5c6332e8-7f15-4ce4-a798-677414c61a88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:25.468874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:25.469763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:25.470746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 242/489] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:25.471696 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6332e8-7f15-4ce4-a798-677414c61a88 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:25.473256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:25.568369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:25.576739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91ab2d-8467-4bf7-9d49-031b41996255 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:25.588969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6332e8-7f15-4ce4-a798-677414c61a88 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:25.621024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a76a90-9bf2-4c32-9c86-b04fa4cd31ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:25.621574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a76a90-9bf2-4c32-9c86-b04fa4cd31ce None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:25.623651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a76a90-9bf2-4c32-9c86-b04fa4cd31ce None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:25.623651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a76a90-9bf2-4c32-9c86-b04fa4cd31ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:25.630507 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac299bb-f52a-4c4d-b6ab-6a72efe1efb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:25.631329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac299bb-f52a-4c4d-b6ab-6a72efe1efb9 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:25.631684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac299bb-f52a-4c4d-b6ab-6a72efe1efb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:25.632026 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac299bb-f52a-4c4d-b6ab-6a72efe1efb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:25.633772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a76a90-9bf2-4c32-9c86-b04fa4cd31ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-35a76a90-9bf2-4c32-9c86-b04fa4cd31ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:25.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:25.641577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:25.642687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 248/490] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:25.644187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac299bb-f52a-4c4d-b6ab-6a72efe1efb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:25.646655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac299bb-f52a-4c4d-b6ab-6a72efe1efb9 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:25.647571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a76a90-9bf2-4c32-9c86-b04fa4cd31ce None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-5ac299bb-f52a-4c4d-b6ab-6a72efe1efb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:25.663643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:25.664750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:25.665793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:25.665793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:25.665793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:25.666878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 243/491] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:25.667734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac299bb-f52a-4c4d-b6ab-6a72efe1efb9 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:25.726215 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:25.760274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a76a90-9bf2-4c32-9c86-b04fa4cd31ce None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:25.780494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac299bb-f52a-4c4d-b6ab-6a72efe1efb9 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:25.799585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d3ef8-c81b-4a39-97e2-02b3f0178544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:25.800475 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d3ef8-c81b-4a39-97e2-02b3f0178544 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:25.801148 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d3ef8-c81b-4a39-97e2-02b3f0178544 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:25.801708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d3ef8-c81b-4a39-97e2-02b3f0178544 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:25.802603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d3ef8-c81b-4a39-97e2-02b3f0178544 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:25.803089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d3ef8-c81b-4a39-97e2-02b3f0178544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:25.804725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d3ef8-c81b-4a39-97e2-02b3f0178544 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:25.814850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:25.819376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 249/492] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:25.822463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d3ef8-c81b-4a39-97e2-02b3f0178544 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:25.831827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519da48-8439-4742-a569-c52bc596b573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:25.832532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519da48-8439-4742-a569-c52bc596b573 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:25.832812 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519da48-8439-4742-a569-c52bc596b573 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:25.833126 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519da48-8439-4742-a569-c52bc596b573 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:25.833871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519da48-8439-4742-a569-c52bc596b573 None admin] RBAC: Adding query filter params (name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:25.834259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519da48-8439-4742-a569-c52bc596b573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:25.837766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519da48-8439-4742-a569-c52bc596b573 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:25.850621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 244/493] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:25.851304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519da48-8439-4742-a569-c52bc596b573 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:25.953668 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d3ef8-c81b-4a39-97e2-02b3f0178544 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:25.974705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519da48-8439-4742-a569-c52bc596b573 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:25.983908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:25.990225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2974dc1-2687-493a-8202-865775d1c84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:25.996160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2974dc1-2687-493a-8202-865775d1c84e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:25.996160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2974dc1-2687-493a-8202-865775d1c84e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:25.996160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2974dc1-2687-493a-8202-865775d1c84e None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:25.996160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2974dc1-2687-493a-8202-865775d1c84e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:25.996160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2974dc1-2687-493a-8202-865775d1c84e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:25.996160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2974dc1-2687-493a-8202-865775d1c84e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:26.007585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 250/494] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:26.008804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2974dc1-2687-493a-8202-865775d1c84e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:26.016019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146c6d5-6c1c-4486-b11b-3098940c99d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:26.016999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146c6d5-6c1c-4486-b11b-3098940c99d9 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.017932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146c6d5-6c1c-4486-b11b-3098940c99d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.018486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146c6d5-6c1c-4486-b11b-3098940c99d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.033691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146c6d5-6c1c-4486-b11b-3098940c99d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:26.036137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146c6d5-6c1c-4486-b11b-3098940c99d9 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-3146c6d5-6c1c-4486-b11b-3098940c99d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:26.053183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:26.054260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:26.055170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:26.055170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:26.055170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:26.056756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 245/495] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:26.058139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61859-d387-494f-a282-3abad6934055 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.058623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61859-d387-494f-a282-3abad6934055 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.059057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61859-d387-494f-a282-3abad6934055 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.060489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 246/496] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:26.061337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d61859-d387-494f-a282-3abad6934055 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:26.121627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:26.143938 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2974dc1-2687-493a-8202-865775d1c84e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:26.180840 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623e175-29bf-4094-803d-72293ea4310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:26.181520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623e175-29bf-4094-803d-72293ea4310c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.181896 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623e175-29bf-4094-803d-72293ea4310c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.182234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623e175-29bf-4094-803d-72293ea4310c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.182339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d61859-d387-494f-a282-3abad6934055 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:26.193748 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623e175-29bf-4094-803d-72293ea4310c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:26.195337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623e175-29bf-4094-803d-72293ea4310c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-9623e175-29bf-4094-803d-72293ea4310c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:26.211233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:26.212160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:26.213010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:26.213010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:26.213010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:26.214570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 251/497] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:26.215933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623e175-29bf-4094-803d-72293ea4310c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:26.221260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09cd64e-869b-443c-9629-ad654d4c0c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:26.221916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cd64e-869b-443c-9629-ad654d4c0c7d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.222198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cd64e-869b-443c-9629-ad654d4c0c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.222503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cd64e-869b-443c-9629-ad654d4c0c7d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.231490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09cd64e-869b-443c-9629-ad654d4c0c7d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:26.232951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09cd64e-869b-443c-9629-ad654d4c0c7d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-c09cd64e-869b-443c-9629-ad654d4c0c7d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:26.249825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:26.251146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:26.252042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:26.252042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:26.252042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:26.252042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:26.252042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:26.252042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:26.252042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:26.252042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:26.252042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:30:26.252042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:26.256774 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 247/498] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Aug 29 23:30:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:26.258095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792660a1-fbe1-48ab-8e73-eb4981632fa2 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.258379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792660a1-fbe1-48ab-8e73-eb4981632fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.258676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792660a1-fbe1-48ab-8e73-eb4981632fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.269436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:26.279125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-792660a1-fbe1-48ab-8e73-eb4981632fa2 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:26.280359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-792660a1-fbe1-48ab-8e73-eb4981632fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:26.331677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623e175-29bf-4094-803d-72293ea4310c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:26.361453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 248/499] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:26.362410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792660a1-fbe1-48ab-8e73-eb4981632fa2 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:26.380451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707eec8-cdfe-4acf-b1d8-1aa1d47641d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:26.381194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707eec8-cdfe-4acf-b1d8-1aa1d47641d2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.381474 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707eec8-cdfe-4acf-b1d8-1aa1d47641d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.381744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707eec8-cdfe-4acf-b1d8-1aa1d47641d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.382390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707eec8-cdfe-4acf-b1d8-1aa1d47641d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:26.382726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707eec8-cdfe-4acf-b1d8-1aa1d47641d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:26.384095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707eec8-cdfe-4acf-b1d8-1aa1d47641d2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:26.396977 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 252/500] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:26.397826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707eec8-cdfe-4acf-b1d8-1aa1d47641d2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:26.412612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:26.482736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792660a1-fbe1-48ab-8e73-eb4981632fa2 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:26.510049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707eec8-cdfe-4acf-b1d8-1aa1d47641d2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:26.514276 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:26.519009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bbd5b-9dc6-4a74-ae0d-d6fd0ab05c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:26.520493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bbd5b-9dc6-4a74-ae0d-d6fd0ab05c12 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.520767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bbd5b-9dc6-4a74-ae0d-d6fd0ab05c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.521033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bbd5b-9dc6-4a74-ae0d-d6fd0ab05c12 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.521716 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bbd5b-9dc6-4a74-ae0d-d6fd0ab05c12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:26.522014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bbd5b-9dc6-4a74-ae0d-d6fd0ab05c12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:26.523411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bbd5b-9dc6-4a74-ae0d-d6fd0ab05c12 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:26.535237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 249/501] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:26.537121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea81690-2d48-4aa8-ba7d-009a02c9d3dd None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.537398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea81690-2d48-4aa8-ba7d-009a02c9d3dd None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.537714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea81690-2d48-4aa8-ba7d-009a02c9d3dd None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.542492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 250/502] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:26.542492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea81690-2d48-4aa8-ba7d-009a02c9d3dd None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:26.558794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff00901-282f-4eff-8255-adf23de3250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:26.559493 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff00901-282f-4eff-8255-adf23de3250c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.559845 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff00901-282f-4eff-8255-adf23de3250c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.560184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff00901-282f-4eff-8255-adf23de3250c None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.561195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff00901-282f-4eff-8255-adf23de3250c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:26.562466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff00901-282f-4eff-8255-adf23de3250c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:26.563363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff00901-282f-4eff-8255-adf23de3250c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:26.606849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 253/503] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Aug 29 23:30:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:26.606849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff00901-282f-4eff-8255-adf23de3250c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:26.653392 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:26.659912 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea81690-2d48-4aa8-ba7d-009a02c9d3dd None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:26.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42ab16-9e2f-425d-a85b-8b0f983a8d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:26.699102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42ab16-9e2f-425d-a85b-8b0f983a8d51 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.699412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42ab16-9e2f-425d-a85b-8b0f983a8d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.699663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42ab16-9e2f-425d-a85b-8b0f983a8d51 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.708183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42ab16-9e2f-425d-a85b-8b0f983a8d51 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:26.709611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42ab16-9e2f-425d-a85b-8b0f983a8d51 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:26.710614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff00901-282f-4eff-8255-adf23de3250c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-3b42ab16-9e2f-425d-a85b-8b0f983a8d51 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:26.720757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:26.721682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:26.723565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:26.723565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:26.723565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:26.723565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:26.723565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:26.723565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:26.723565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:26.723565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:26.723565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 23:30:26.723565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:26.724348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 251/504] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:26.725649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e283ba7-d295-4c2b-99fa-0d057b276780 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.725944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e283ba7-d295-4c2b-99fa-0d057b276780 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.726224 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e283ba7-d295-4c2b-99fa-0d057b276780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.743963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-9e283ba7-d295-4c2b-99fa-0d057b276780 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:26.745073 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9e283ba7-d295-4c2b-99fa-0d057b276780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:26.747168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182e574-6b70-4f40-b949-4052c826ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:26.747793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182e574-6b70-4f40-b949-4052c826ec32 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.748064 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182e574-6b70-4f40-b949-4052c826ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.748328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182e574-6b70-4f40-b949-4052c826ec32 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.755585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182e574-6b70-4f40-b949-4052c826ec32 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:26.757032 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182e574-6b70-4f40-b949-4052c826ec32 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a182e574-6b70-4f40-b949-4052c826ec32 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:26.770832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:26.771782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:26.772690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:26.772690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:26.772690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:26.772690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:26.772690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:26.772690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:26.772690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:26.772690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:26.772690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 23:30:26.772690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:26.772690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 254/505] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:26.773255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182e574-6b70-4f40-b949-4052c826ec32 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:26.800844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:26.817669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 252/506] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:26.818567 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e283ba7-d295-4c2b-99fa-0d057b276780 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:26.882404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182e574-6b70-4f40-b949-4052c826ec32 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:26.913613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:26.920004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f836e9-f62d-46d0-9229-6e94984e1f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:26.920610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f836e9-f62d-46d0-9229-6e94984e1f1b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.920916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f836e9-f62d-46d0-9229-6e94984e1f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.921161 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f836e9-f62d-46d0-9229-6e94984e1f1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.931071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e283ba7-d295-4c2b-99fa-0d057b276780 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:26.973310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf6625-e4f8-409f-8c62-ad00bba8747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:26.973952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf6625-e4f8-409f-8c62-ad00bba8747d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:26.974239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf6625-e4f8-409f-8c62-ad00bba8747d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:26.974561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf6625-e4f8-409f-8c62-ad00bba8747d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:26.989205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf6625-e4f8-409f-8c62-ad00bba8747d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-a8bf6625-e4f8-409f-8c62-ad00bba8747d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:26.995045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:26.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:26.996741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 253/507] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:26.997443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf6625-e4f8-409f-8c62-ad00bba8747d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:27.011517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f836e9-f62d-46d0-9229-6e94984e1f1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:27.013008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f836e9-f62d-46d0-9229-6e94984e1f1b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:27.039043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:27.117522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf6625-e4f8-409f-8c62-ad00bba8747d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:27.126262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 255/508] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:27.127196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f836e9-f62d-46d0-9229-6e94984e1f1b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:27.152989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d263b-401f-489f-9634-f015a998d4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:27.153586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d263b-401f-489f-9634-f015a998d4bc None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:27.153867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d263b-401f-489f-9634-f015a998d4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:27.154134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d263b-401f-489f-9634-f015a998d4bc None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:27.154890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d263b-401f-489f-9634-f015a998d4bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:27.155195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d263b-401f-489f-9634-f015a998d4bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:27.156613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d263b-401f-489f-9634-f015a998d4bc None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:27.194689 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 254/509] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Aug 29 23:30:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:27.196181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d263b-401f-489f-9634-f015a998d4bc None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:27.199621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:27.234805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f836e9-f62d-46d0-9229-6e94984e1f1b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:27.259118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:27.272362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9f01e-30ac-4318-a2c6-d1c21cc242ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:27.272983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9f01e-30ac-4318-a2c6-d1c21cc242ce None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:27.273271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9f01e-30ac-4318-a2c6-d1c21cc242ce None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:27.273564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9f01e-30ac-4318-a2c6-d1c21cc242ce None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:27.276124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9f01e-30ac-4318-a2c6-d1c21cc242ce None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:27.276124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9f01e-30ac-4318-a2c6-d1c21cc242ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:27.276124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9f01e-30ac-4318-a2c6-d1c21cc242ce None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:27.294738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 256/510] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Aug 29 23:30:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:27.295625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9f01e-30ac-4318-a2c6-d1c21cc242ce None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:27.310755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d263b-401f-489f-9634-f015a998d4bc None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:27.355570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fece1-1c6e-4037-83ec-6cfe8c74358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:27.356185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fece1-1c6e-4037-83ec-6cfe8c74358c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:27.356460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fece1-1c6e-4037-83ec-6cfe8c74358c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:27.356727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fece1-1c6e-4037-83ec-6cfe8c74358c None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:27.357403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fece1-1c6e-4037-83ec-6cfe8c74358c None admin] RBAC: Adding query filter params (name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:27.357702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fece1-1c6e-4037-83ec-6cfe8c74358c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:27.359144 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fece1-1c6e-4037-83ec-6cfe8c74358c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:27.368012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 255/511] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:27.368896 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fece1-1c6e-4037-83ec-6cfe8c74358c None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:27.412586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9f01e-30ac-4318-a2c6-d1c21cc242ce None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:27.443992 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:27.452224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9feebe-f473-4cba-8973-5ecab38e6107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:27.452871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9feebe-f473-4cba-8973-5ecab38e6107 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:27.453151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9feebe-f473-4cba-8973-5ecab38e6107 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:27.453423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9feebe-f473-4cba-8973-5ecab38e6107 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:27.467735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9feebe-f473-4cba-8973-5ecab38e6107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:27.472495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9feebe-f473-4cba-8973-5ecab38e6107 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:27.480377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fece1-1c6e-4037-83ec-6cfe8c74358c None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-6c9feebe-f473-4cba-8973-5ecab38e6107 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:27.486009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:27.487156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:27.496168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:27.496168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:27.496168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:27.496168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 257/512] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:27.496168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9feebe-f473-4cba-8973-5ecab38e6107 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:27.523550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:27.529553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe337a0-f090-428d-ace5-636c7cdce1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:27.530175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe337a0-f090-428d-ace5-636c7cdce1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:27.530492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe337a0-f090-428d-ace5-636c7cdce1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:27.530766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe337a0-f090-428d-ace5-636c7cdce1b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:27.544773 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe337a0-f090-428d-ace5-636c7cdce1b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:27.546245 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe337a0-f090-428d-ace5-636c7cdce1b5 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-cbe337a0-f090-428d-ace5-636c7cdce1b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:27.565204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:27.567017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:27.568131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:27.568131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:27.568131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:27.568468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 256/513] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:27.569377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe337a0-f090-428d-ace5-636c7cdce1b5 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:27.603315 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9feebe-f473-4cba-8973-5ecab38e6107 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:27.639768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dfdf2-12ab-4d86-b411-d8cdd98b7cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:27.640680 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dfdf2-12ab-4d86-b411-d8cdd98b7cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:27.640842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dfdf2-12ab-4d86-b411-d8cdd98b7cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:27.641265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dfdf2-12ab-4d86-b411-d8cdd98b7cc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:27.693850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe337a0-f090-428d-ace5-636c7cdce1b5 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:27.712310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:27.715874 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85dfdf2-12ab-4d86-b411-d8cdd98b7cc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:27.717372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85dfdf2-12ab-4d86-b411-d8cdd98b7cc3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:27.730143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082331e-b858-4d3c-bfd8-a6f9534c8430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:27.730784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082331e-b858-4d3c-bfd8-a6f9534c8430 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:27.731056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082331e-b858-4d3c-bfd8-a6f9534c8430 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:27.731332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082331e-b858-4d3c-bfd8-a6f9534c8430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:27.750478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:27.805637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082331e-b858-4d3c-bfd8-a6f9534c8430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:27.807126 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082331e-b858-4d3c-bfd8-a6f9534c8430 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:27.823898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 258/514] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:27.829278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dfdf2-12ab-4d86-b411-d8cdd98b7cc3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:27.924129 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 257/515] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:27.927368 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082331e-b858-4d3c-bfd8-a6f9534c8430 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:27.936040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dfdf2-12ab-4d86-b411-d8cdd98b7cc3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:27.949456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:27.974365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:27.979360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806bae4-1e31-4861-b09b-2e3067082687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:27.979965 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806bae4-1e31-4861-b09b-2e3067082687 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:27.980243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806bae4-1e31-4861-b09b-2e3067082687 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:27.980521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806bae4-1e31-4861-b09b-2e3067082687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:27.991242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806bae4-1e31-4861-b09b-2e3067082687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:27.992704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806bae4-1e31-4861-b09b-2e3067082687 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-2806bae4-1e31-4861-b09b-2e3067082687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:28.008143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:28.009287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:28.010163 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:28.010163 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:28.010163 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:28.010163 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 259/516] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:28.010378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806bae4-1e31-4861-b09b-2e3067082687 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:28.039973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082331e-b858-4d3c-bfd8-a6f9534c8430 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:28.089194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b3671-8ba8-420e-9ba0-8984ad621b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:28.089719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b3671-8ba8-420e-9ba0-8984ad621b80 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:28.089996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b3671-8ba8-420e-9ba0-8984ad621b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:28.090267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b3671-8ba8-420e-9ba0-8984ad621b80 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:28.090953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b3671-8ba8-420e-9ba0-8984ad621b80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:28.091259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b3671-8ba8-420e-9ba0-8984ad621b80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:28.092664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b3671-8ba8-420e-9ba0-8984ad621b80 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:28.107205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 258/517] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:28.108062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b3671-8ba8-420e-9ba0-8984ad621b80 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:28.122602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806bae4-1e31-4861-b09b-2e3067082687 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:28.156331 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5529578-03ca-4ca4-8294-ceba257e8b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:28.156937 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5529578-03ca-4ca4-8294-ceba257e8b02 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:28.157216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5529578-03ca-4ca4-8294-ceba257e8b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:28.157489 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5529578-03ca-4ca4-8294-ceba257e8b02 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:28.158126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5529578-03ca-4ca4-8294-ceba257e8b02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:28.158454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5529578-03ca-4ca4-8294-ceba257e8b02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:28.160416 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5529578-03ca-4ca4-8294-ceba257e8b02 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:28.172769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 260/518] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:28.173617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5529578-03ca-4ca4-8294-ceba257e8b02 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:28.189747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:28.211976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b3671-8ba8-420e-9ba0-8984ad621b80 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:28.222172 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:28.247145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d6402-97c5-473b-8a88-fb89550a63a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:28.247775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d6402-97c5-473b-8a88-fb89550a63a8 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:28.248057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d6402-97c5-473b-8a88-fb89550a63a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:28.248328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d6402-97c5-473b-8a88-fb89550a63a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:28.259272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d6402-97c5-473b-8a88-fb89550a63a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:28.260733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d6402-97c5-473b-8a88-fb89550a63a8 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-d87d6402-97c5-473b-8a88-fb89550a63a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:28.278091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:28.279496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:28.281571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:28.281571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:28.281571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:28.281571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 259/519] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:28.281571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d6402-97c5-473b-8a88-fb89550a63a8 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:28.288924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5529578-03ca-4ca4-8294-ceba257e8b02 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:28.326737 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd028021-fc0d-456c-a446-cb66ddd6804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:28.327375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028021-fc0d-456c-a446-cb66ddd6804a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:28.327650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028021-fc0d-456c-a446-cb66ddd6804a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:28.327909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028021-fc0d-456c-a446-cb66ddd6804a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:28.339587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd028021-fc0d-456c-a446-cb66ddd6804a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:28.340976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd028021-fc0d-456c-a446-cb66ddd6804a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-fd028021-fc0d-456c-a446-cb66ddd6804a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:28.357304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:28.358388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:28.359376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:28.359376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:28.359376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:28.359376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 261/520] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:28.359376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd028021-fc0d-456c-a446-cb66ddd6804a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:28.393645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d6402-97c5-473b-8a88-fb89550a63a8 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:28.424292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:28.431070 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35ce62-0951-4970-b6dd-0457dce5c2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:28.431681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35ce62-0951-4970-b6dd-0457dce5c2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:28.433737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35ce62-0951-4970-b6dd-0457dce5c2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:28.433737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35ce62-0951-4970-b6dd-0457dce5c2d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:28.433737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35ce62-0951-4970-b6dd-0457dce5c2d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:28.433737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35ce62-0951-4970-b6dd-0457dce5c2d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:28.434558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35ce62-0951-4970-b6dd-0457dce5c2d5 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:28.447657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 260/521] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:28.448510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35ce62-0951-4970-b6dd-0457dce5c2d5 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:28.465862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:28.471793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd028021-fc0d-456c-a446-cb66ddd6804a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:28.521316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627cead-8fe1-4659-b5a5-6f3c128e3eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:28.522252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627cead-8fe1-4659-b5a5-6f3c128e3eff None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:28.522575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627cead-8fe1-4659-b5a5-6f3c128e3eff None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:28.522856 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627cead-8fe1-4659-b5a5-6f3c128e3eff None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:28.523530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627cead-8fe1-4659-b5a5-6f3c128e3eff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:28.525001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627cead-8fe1-4659-b5a5-6f3c128e3eff None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:28.571609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35ce62-0951-4970-b6dd-0457dce5c2d5 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:28.590220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 262/522] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:30:28 2022] POST /identity/v3/users => generated 247 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:30:28.591794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627cead-8fe1-4659-b5a5-6f3c128e3eff None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:28.630688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2e3ea-3523-4ea3-b69a-f1fa4df35d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:28.631344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2e3ea-3523-4ea3-b69a-f1fa4df35d51 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:28.631627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2e3ea-3523-4ea3-b69a-f1fa4df35d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:28.631903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2e3ea-3523-4ea3-b69a-f1fa4df35d51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:28.639619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2e3ea-3523-4ea3-b69a-f1fa4df35d51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:28.641121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2e3ea-3523-4ea3-b69a-f1fa4df35d51 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-ddb2e3ea-3523-4ea3-b69a-f1fa4df35d51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:28.655828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:28.656800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:28.657778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:28.657778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:28.657778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:28.657778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:28.657778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:28.657778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:28.657778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:28.657778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:28.662587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 261/523] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:28.663573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2e3ea-3523-4ea3-b69a-f1fa4df35d51 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:28.724433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:28.738088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627cead-8fe1-4659-b5a5-6f3c128e3eff None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:28.778477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db076a00-dec8-4961-857f-6e4134c1753d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:28.779496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db076a00-dec8-4961-857f-6e4134c1753d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:28.779787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db076a00-dec8-4961-857f-6e4134c1753d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:28.780055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db076a00-dec8-4961-857f-6e4134c1753d None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:28.780737 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db076a00-dec8-4961-857f-6e4134c1753d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:28.781043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db076a00-dec8-4961-857f-6e4134c1753d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:28.782455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db076a00-dec8-4961-857f-6e4134c1753d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:28.788853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2e3ea-3523-4ea3-b69a-f1fa4df35d51 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:28.797276 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 263/524] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:28.797939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db076a00-dec8-4961-857f-6e4134c1753d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:28.814208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:28.837130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88e591-25eb-4baf-aefa-d6caf228025b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:28.837744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88e591-25eb-4baf-aefa-d6caf228025b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:28.838037 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88e591-25eb-4baf-aefa-d6caf228025b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:28.838309 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88e591-25eb-4baf-aefa-d6caf228025b None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:28.839157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88e591-25eb-4baf-aefa-d6caf228025b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:28.839466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88e591-25eb-4baf-aefa-d6caf228025b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:28.841799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88e591-25eb-4baf-aefa-d6caf228025b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:28.862410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 262/525] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:28.862410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88e591-25eb-4baf-aefa-d6caf228025b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:28.908969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db076a00-dec8-4961-857f-6e4134c1753d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:28.951056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8cbb2-af1e-4b6a-90d9-11f17b480806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:28.951898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8cbb2-af1e-4b6a-90d9-11f17b480806 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:28.952277 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8cbb2-af1e-4b6a-90d9-11f17b480806 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:28.953089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8cbb2-af1e-4b6a-90d9-11f17b480806 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:28.963039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8cbb2-af1e-4b6a-90d9-11f17b480806 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:28.964206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8cbb2-af1e-4b6a-90d9-11f17b480806 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-21c8cbb2-af1e-4b6a-90d9-11f17b480806 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:28.979070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:28.980320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:28.981473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:28.981473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:28.981473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:28.981473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:28.981473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:28.981473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:28.981473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:28.981473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:28.981473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 23:30:28.981473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:28.982273 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 264/526] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:28.989015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8cbb2-af1e-4b6a-90d9-11f17b480806 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:28.993318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88e591-25eb-4baf-aefa-d6caf228025b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:29.005694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:29.030731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865ba68-d3bb-4dce-b07b-6db28ea49eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:29.031434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865ba68-d3bb-4dce-b07b-6db28ea49eab None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:29.031842 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865ba68-d3bb-4dce-b07b-6db28ea49eab None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:29.032185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865ba68-d3bb-4dce-b07b-6db28ea49eab None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:29.032959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865ba68-d3bb-4dce-b07b-6db28ea49eab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:29.033333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865ba68-d3bb-4dce-b07b-6db28ea49eab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:29.034850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865ba68-d3bb-4dce-b07b-6db28ea49eab None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:29.063294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 263/527] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:29.063294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865ba68-d3bb-4dce-b07b-6db28ea49eab None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:29.083219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:29.133622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8cbb2-af1e-4b6a-90d9-11f17b480806 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:29.178967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e7041-9c3a-4430-8847-c01896f18187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:29.179597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e7041-9c3a-4430-8847-c01896f18187 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:29.179875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e7041-9c3a-4430-8847-c01896f18187 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:29.180144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e7041-9c3a-4430-8847-c01896f18187 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:29.187174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e7041-9c3a-4430-8847-c01896f18187 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:29.188637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e7041-9c3a-4430-8847-c01896f18187 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:29.200794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865ba68-d3bb-4dce-b07b-6db28ea49eab None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ee4e7041-9c3a-4430-8847-c01896f18187 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:29.211978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:29.212872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:29.213716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:29.213716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:29.213716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:29.213716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:29.213716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:29.213716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:29.213716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:29.213716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:29.214120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 265/528] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:29.215431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e7041-9c3a-4430-8847-c01896f18187 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:29.248636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41921d7a-3572-41fa-84c2-c7831b195715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:29.250595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921d7a-3572-41fa-84c2-c7831b195715 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:29.251056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921d7a-3572-41fa-84c2-c7831b195715 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:29.251457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921d7a-3572-41fa-84c2-c7831b195715 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:29.267341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41921d7a-3572-41fa-84c2-c7831b195715 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:29.268829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41921d7a-3572-41fa-84c2-c7831b195715 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-41921d7a-3572-41fa-84c2-c7831b195715 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:29.281856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:29.283014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:29.284047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:29.284047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:29.284047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:29.284047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:29.284047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:29.284047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:29.284047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:29.284047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:29.285783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 264/529] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:29.286198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41921d7a-3572-41fa-84c2-c7831b195715 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:29.322663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:29.342626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e7041-9c3a-4430-8847-c01896f18187 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:29.379926 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:29.384856 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835894e-81aa-4492-8596-f701572196b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:29.385397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835894e-81aa-4492-8596-f701572196b0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:29.385679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835894e-81aa-4492-8596-f701572196b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:29.385947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835894e-81aa-4492-8596-f701572196b0 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:29.386696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835894e-81aa-4492-8596-f701572196b0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:29.387002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835894e-81aa-4492-8596-f701572196b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:29.388433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835894e-81aa-4492-8596-f701572196b0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:29.409348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41921d7a-3572-41fa-84c2-c7831b195715 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:29.413830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 266/530] 10.222.0.58 () {56 vars in 1139 bytes} [Mon Aug 29 23:30:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:29.415327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835894e-81aa-4492-8596-f701572196b0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:29.453278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1234751-b1e6-4851-91e8-e475d8e1b219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:29.454163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1234751-b1e6-4851-91e8-e475d8e1b219 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:29.454571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1234751-b1e6-4851-91e8-e475d8e1b219 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:29.455124 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1234751-b1e6-4851-91e8-e475d8e1b219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:29.456577 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1234751-b1e6-4851-91e8-e475d8e1b219 None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=e4a8261d92064b838fc6e8f415879566) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:29.457400 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1234751-b1e6-4851-91e8-e475d8e1b219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:29.459497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1234751-b1e6-4851-91e8-e475d8e1b219 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:29.466790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 265/531] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:30:29 2022] GET /identity/v3/role_assignments?user.id=e4a8261d92064b838fc6e8f415879566&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:29.467976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1234751-b1e6-4851-91e8-e475d8e1b219 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:29.562366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835894e-81aa-4492-8596-f701572196b0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:29.610205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:29.614224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf2d5c-26f2-4751-9011-356042e870be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:29.615094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf2d5c-26f2-4751-9011-356042e870be None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:29.615502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf2d5c-26f2-4751-9011-356042e870be None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:29.615865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf2d5c-26f2-4751-9011-356042e870be None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:29.616855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf2d5c-26f2-4751-9011-356042e870be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:29.617137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf2d5c-26f2-4751-9011-356042e870be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:29.618752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf2d5c-26f2-4751-9011-356042e870be None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:29.620191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1234751-b1e6-4851-91e8-e475d8e1b219 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:29.640574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 267/532] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:29.643062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf2d5c-26f2-4751-9011-356042e870be None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:29.665750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f1786-b381-44e8-81be-f9429fd11b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:29.666746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f1786-b381-44e8-81be-f9429fd11b41 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:29.667579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f1786-b381-44e8-81be-f9429fd11b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:29.669408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f1786-b381-44e8-81be-f9429fd11b41 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:29.670671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f1786-b381-44e8-81be-f9429fd11b41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:29.671428 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f1786-b381-44e8-81be-f9429fd11b41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:29.673780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f1786-b381-44e8-81be-f9429fd11b41 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:29.696999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 266/533] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:29.696999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f1786-b381-44e8-81be-f9429fd11b41 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:29.722604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:29.789475 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf2d5c-26f2-4751-9011-356042e870be None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:29.835678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a49703-d521-45f4-a960-e6aa0aae11af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:29.836512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a49703-d521-45f4-a960-e6aa0aae11af None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:29.837123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a49703-d521-45f4-a960-e6aa0aae11af None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:29.837237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a49703-d521-45f4-a960-e6aa0aae11af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:29.843600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f1786-b381-44e8-81be-f9429fd11b41 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:29.882967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f17e59-775c-451d-b949-21ee14cb44bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:29.883816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f17e59-775c-451d-b949-21ee14cb44bd None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:29.884232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f17e59-775c-451d-b949-21ee14cb44bd None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:29.884639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f17e59-775c-451d-b949-21ee14cb44bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:29.885642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f17e59-775c-451d-b949-21ee14cb44bd None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=d7a59337d20e4678b8a16ac35bffbdb4) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:29.886091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f17e59-775c-451d-b949-21ee14cb44bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:29.887791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f17e59-775c-451d-b949-21ee14cb44bd None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:29.895696 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 267/534] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:30:29 2022] GET /identity/v3/role_assignments?user.id=d7a59337d20e4678b8a16ac35bffbdb4&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:29.896911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f17e59-775c-451d-b949-21ee14cb44bd None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:29.920540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:29.949683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a49703-d521-45f4-a960-e6aa0aae11af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:29.953964 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a49703-d521-45f4-a960-e6aa0aae11af None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:30.060682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:30.111484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f17e59-775c-451d-b949-21ee14cb44bd None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:30.156911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 268/535] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:30.164889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a49703-d521-45f4-a960-e6aa0aae11af None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:30.174900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d2976-33d0-454d-af3b-3dc6241a373a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:30.176361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d2976-33d0-454d-af3b-3dc6241a373a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:30.176771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d2976-33d0-454d-af3b-3dc6241a373a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:30.177152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d2976-33d0-454d-af3b-3dc6241a373a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:30.178200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d2976-33d0-454d-af3b-3dc6241a373a None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=64a42591a6814a0cbcb8444829cb25b3) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:30.178666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d2976-33d0-454d-af3b-3dc6241a373a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:30.180371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d2976-33d0-454d-af3b-3dc6241a373a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:30.197507 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 268/536] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:30:29 2022] GET /identity/v3/role_assignments?user.id=64a42591a6814a0cbcb8444829cb25b3&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:30.340116 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a49703-d521-45f4-a960-e6aa0aae11af None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:30.382612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bdceeb-5630-4590-8ebe-1b18291a5879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:30.383232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdceeb-5630-4590-8ebe-1b18291a5879 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:30.383515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdceeb-5630-4590-8ebe-1b18291a5879 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:30.383781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdceeb-5630-4590-8ebe-1b18291a5879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:30.396421 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bdceeb-5630-4590-8ebe-1b18291a5879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:30.398109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bdceeb-5630-4590-8ebe-1b18291a5879 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-34bdceeb-5630-4590-8ebe-1b18291a5879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:30.416373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:30.417298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:30.418143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:30.418143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:30.418143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:30.418143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 269/537] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:30.423311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d2976-33d0-454d-af3b-3dc6241a373a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:30.591410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:30.664167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d2976-33d0-454d-af3b-3dc6241a373a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:30.734542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11750353-065b-4764-8a61-efd9e0aa5e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:30.734542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11750353-065b-4764-8a61-efd9e0aa5e03 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:30.734542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11750353-065b-4764-8a61-efd9e0aa5e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:30.734542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11750353-065b-4764-8a61-efd9e0aa5e03 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:30.738444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11750353-065b-4764-8a61-efd9e0aa5e03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:30.738444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11750353-065b-4764-8a61-efd9e0aa5e03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:30.738662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11750353-065b-4764-8a61-efd9e0aa5e03 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:30.761862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 269/538] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:30.768203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bdceeb-5630-4590-8ebe-1b18291a5879 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:30.773756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:30.890135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea985374-a04e-4157-bf45-5e365a9d1222 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:30.890677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea985374-a04e-4157-bf45-5e365a9d1222 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:30.891161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea985374-a04e-4157-bf45-5e365a9d1222 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:30.893772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 270/539] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:30:30.904059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8125d-8f6f-488b-b8db-38ef5e78f42f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:30.904516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8125d-8f6f-488b-b8db-38ef5e78f42f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:30.904995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8125d-8f6f-488b-b8db-38ef5e78f42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:30.962458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-a2d8125d-8f6f-488b-b8db-38ef5e78f42f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:30.966342 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a2d8125d-8f6f-488b-b8db-38ef5e78f42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:31.092755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bdceeb-5630-4590-8ebe-1b18291a5879 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:31.225799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 271/540] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:30:31.227891 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884dc8d-e279-4a68-9038-0ea13ce677a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:31.228941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884dc8d-e279-4a68-9038-0ea13ce677a5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:31.229429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884dc8d-e279-4a68-9038-0ea13ce677a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:31.229895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884dc8d-e279-4a68-9038-0ea13ce677a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:31.243682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884dc8d-e279-4a68-9038-0ea13ce677a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:31.246459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884dc8d-e279-4a68-9038-0ea13ce677a5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-6884dc8d-e279-4a68-9038-0ea13ce677a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:31.270024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:31.271086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:31.271949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:31.271949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:31.271949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:31.271949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:31.271949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:31.271949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:31.271949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:31.271949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:31.272821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 270/541] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 505 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:31.278894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8125d-8f6f-488b-b8db-38ef5e78f42f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:31.447855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:31.488758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8125d-8f6f-488b-b8db-38ef5e78f42f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:31.618216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac05275-5399-4aa4-ac36-b1c43a8f1170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:31.618793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac05275-5399-4aa4-ac36-b1c43a8f1170 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:31.619074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac05275-5399-4aa4-ac36-b1c43a8f1170 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:31.619346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac05275-5399-4aa4-ac36-b1c43a8f1170 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:31.620062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac05275-5399-4aa4-ac36-b1c43a8f1170 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:31.620369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac05275-5399-4aa4-ac36-b1c43a8f1170 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:31.621817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac05275-5399-4aa4-ac36-b1c43a8f1170 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:31.640712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 272/542] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:31.648080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884dc8d-e279-4a68-9038-0ea13ce677a5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:31.725581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:31.762357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3568-8583-4dee-9616-4864e56e44b3 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:31.762922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3568-8583-4dee-9616-4864e56e44b3 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:31.763340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3568-8583-4dee-9616-4864e56e44b3 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:31.766749 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 273/543] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:31.771766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54ce9e-e5c9-41f9-85dd-3b86a6bfe069 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:31.772208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884dc8d-e279-4a68-9038-0ea13ce677a5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:31.772587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54ce9e-e5c9-41f9-85dd-3b86a6bfe069 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:31.773081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54ce9e-e5c9-41f9-85dd-3b86a6bfe069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:31.792898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-fe54ce9e-e5c9-41f9-85dd-3b86a6bfe069 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:31.794514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-fe54ce9e-e5c9-41f9-85dd-3b86a6bfe069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:31.813070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1f62c-ac02-4d2a-9637-9aa0793e5192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:31.813681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1f62c-ac02-4d2a-9637-9aa0793e5192 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:31.813959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1f62c-ac02-4d2a-9637-9aa0793e5192 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:31.814306 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1f62c-ac02-4d2a-9637-9aa0793e5192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:31.815135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1f62c-ac02-4d2a-9637-9aa0793e5192 None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=863bcfa6b8694182b9d163133a141907) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:31.815445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1f62c-ac02-4d2a-9637-9aa0793e5192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:31.817480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1f62c-ac02-4d2a-9637-9aa0793e5192 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:31.823771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 271/544] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:30:31 2022] GET /identity/v3/role_assignments?user.id=863bcfa6b8694182b9d163133a141907&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:31.919177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1f62c-ac02-4d2a-9637-9aa0793e5192 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:31.920832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 274/545] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:30:32.083667 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1f62c-ac02-4d2a-9637-9aa0793e5192 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:32.138448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:32.147307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bbdcd-cb88-49a6-9770-01ef5a833a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:32.148255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bbdcd-cb88-49a6-9770-01ef5a833a97 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:32.150830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bbdcd-cb88-49a6-9770-01ef5a833a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:32.150830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bbdcd-cb88-49a6-9770-01ef5a833a97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:32.168673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bbdcd-cb88-49a6-9770-01ef5a833a97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-175bbdcd-cb88-49a6-9770-01ef5a833a97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:32.173865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:32.174741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:32.175582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:32.182447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 272/546] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:32.186839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54ce9e-e5c9-41f9-85dd-3b86a6bfe069 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:32.333286 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:32.402519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54ce9e-e5c9-41f9-85dd-3b86a6bfe069 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:32.411772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bbdcd-cb88-49a6-9770-01ef5a833a97 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:32.458856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58818ad-2492-4799-b7bd-6b1c00b6871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:32.460115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58818ad-2492-4799-b7bd-6b1c00b6871d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:32.461149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58818ad-2492-4799-b7bd-6b1c00b6871d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:32.461709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58818ad-2492-4799-b7bd-6b1c00b6871d None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:32.462634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58818ad-2492-4799-b7bd-6b1c00b6871d None admin] RBAC: Adding query filter params (name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:32.470462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58818ad-2492-4799-b7bd-6b1c00b6871d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:32.470462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58818ad-2492-4799-b7bd-6b1c00b6871d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:32.477135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 275/547] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:32.486747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58818ad-2492-4799-b7bd-6b1c00b6871d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:32.656085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bbdcd-cb88-49a6-9770-01ef5a833a97 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:32.714796 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58818ad-2492-4799-b7bd-6b1c00b6871d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:32.738970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370d82a-b477-499c-9aa8-3fc469d942e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:32.739611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370d82a-b477-499c-9aa8-3fc469d942e7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:32.739888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370d82a-b477-499c-9aa8-3fc469d942e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:32.740157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370d82a-b477-499c-9aa8-3fc469d942e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:32.767035 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370d82a-b477-499c-9aa8-3fc469d942e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c370d82a-b477-499c-9aa8-3fc469d942e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:32.774802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:32.775733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:32.777016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:32.778227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:32.782871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 273/548] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:32.784576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370d82a-b477-499c-9aa8-3fc469d942e7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:32.800721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1cc39-1e2e-4fba-ac50-304ec0a430e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:32.802376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1cc39-1e2e-4fba-ac50-304ec0a430e0 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:32.802376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1cc39-1e2e-4fba-ac50-304ec0a430e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:32.802376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1cc39-1e2e-4fba-ac50-304ec0a430e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:32.827123 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1cc39-1e2e-4fba-ac50-304ec0a430e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:32.828580 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1cc39-1e2e-4fba-ac50-304ec0a430e0 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-4af1cc39-1e2e-4fba-ac50-304ec0a430e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:32.860889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:32.861871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:32.863737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:32.863737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:32.863737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:32.863737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 276/549] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:32.868152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa01cc-668b-4eac-8a1c-776685d090ed None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:32.868152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa01cc-668b-4eac-8a1c-776685d090ed None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:32.868399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa01cc-668b-4eac-8a1c-776685d090ed None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:32.873920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 277/550] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:32.873920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa01cc-668b-4eac-8a1c-776685d090ed None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:32.889533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:32.971174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370d82a-b477-499c-9aa8-3fc469d942e7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:33.009414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa01cc-668b-4eac-8a1c-776685d090ed None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:33.024464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea030b2-3acc-449d-aa5d-8ad58ed99e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:33.025107 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea030b2-3acc-449d-aa5d-8ad58ed99e69 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:33.025385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea030b2-3acc-449d-aa5d-8ad58ed99e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:33.025657 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea030b2-3acc-449d-aa5d-8ad58ed99e69 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:33.026333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea030b2-3acc-449d-aa5d-8ad58ed99e69 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:33.026685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea030b2-3acc-449d-aa5d-8ad58ed99e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:33.028082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea030b2-3acc-449d-aa5d-8ad58ed99e69 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:33.042793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 274/551] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:33.042793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940ba37-6da4-4088-bc31-bceedc825a9d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:33.042793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940ba37-6da4-4088-bc31-bceedc825a9d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:33.042793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940ba37-6da4-4088-bc31-bceedc825a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:33.061205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a940ba37-6da4-4088-bc31-bceedc825a9d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:33.061803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f79fb-271f-447a-99f2-896ba4e2b49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:33.062415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a940ba37-6da4-4088-bc31-bceedc825a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:33.063017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f79fb-271f-447a-99f2-896ba4e2b49d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:33.063492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f79fb-271f-447a-99f2-896ba4e2b49d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:33.063961 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f79fb-271f-447a-99f2-896ba4e2b49d None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:33.065149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f79fb-271f-447a-99f2-896ba4e2b49d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:33.066850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f79fb-271f-447a-99f2-896ba4e2b49d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:33.068249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f79fb-271f-447a-99f2-896ba4e2b49d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:33.086171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 278/552] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:33.087400 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f79fb-271f-447a-99f2-896ba4e2b49d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:33.140727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:33.162448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 275/553] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:33.162448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940ba37-6da4-4088-bc31-bceedc825a9d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:33.240310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:33.281513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f79fb-271f-447a-99f2-896ba4e2b49d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:33.380575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940ba37-6da4-4088-bc31-bceedc825a9d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:33.387338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196021db-3888-4bed-bf4d-baeb822968be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:33.387984 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196021db-3888-4bed-bf4d-baeb822968be None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:33.388255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196021db-3888-4bed-bf4d-baeb822968be None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:33.388520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196021db-3888-4bed-bf4d-baeb822968be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:33.408484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196021db-3888-4bed-bf4d-baeb822968be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:33.412536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196021db-3888-4bed-bf4d-baeb822968be None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-196021db-3888-4bed-bf4d-baeb822968be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:33.440189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:33.441782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:33.442778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:33.442778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:33.442778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:33.443178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 279/554] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:33.453152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196021db-3888-4bed-bf4d-baeb822968be None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:33.467187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ac6b3-1dda-4e15-af0c-000de2c78775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:33.468078 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ac6b3-1dda-4e15-af0c-000de2c78775 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:33.468520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ac6b3-1dda-4e15-af0c-000de2c78775 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:33.468932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ac6b3-1dda-4e15-af0c-000de2c78775 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:33.497333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ac6b3-1dda-4e15-af0c-000de2c78775 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:33.500824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ac6b3-1dda-4e15-af0c-000de2c78775 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-8c6ac6b3-1dda-4e15-af0c-000de2c78775 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:33.525663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:33.526704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:33.527601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:33.527601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:33.527601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:33.527601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:33.527601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:33.527601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:33.527601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:33.527601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:33.527601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:30:33.527601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:33.528134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 276/555] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:30:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:33.529898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf1186-a81f-4c44-a536-ca21e8c88eec None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:33.530400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf1186-a81f-4c44-a536-ca21e8c88eec None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:33.531013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf1186-a81f-4c44-a536-ca21e8c88eec None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:33.534465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 277/556] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:33.536039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf1186-a81f-4c44-a536-ca21e8c88eec None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:33.573996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:33.592308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196021db-3888-4bed-bf4d-baeb822968be None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:33.629379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe17b4-ee2d-448c-919e-729bf212c687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:33.629982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe17b4-ee2d-448c-919e-729bf212c687 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:33.630261 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe17b4-ee2d-448c-919e-729bf212c687 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:33.630563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe17b4-ee2d-448c-919e-729bf212c687 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:33.631230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe17b4-ee2d-448c-919e-729bf212c687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:33.631532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe17b4-ee2d-448c-919e-729bf212c687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:33.632926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe17b4-ee2d-448c-919e-729bf212c687 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:33.647559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 280/557] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:33.648294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe17b4-ee2d-448c-919e-729bf212c687 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:33.693331 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:33.705789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf1186-a81f-4c44-a536-ca21e8c88eec None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:33.753434 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5929d44-aa6d-4f95-9bda-3b2ef97e7eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:33.754261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5929d44-aa6d-4f95-9bda-3b2ef97e7eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:33.754721 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5929d44-aa6d-4f95-9bda-3b2ef97e7eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:33.755130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5929d44-aa6d-4f95-9bda-3b2ef97e7eb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:33.774088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5929d44-aa6d-4f95-9bda-3b2ef97e7eb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-b5929d44-aa6d-4f95-9bda-3b2ef97e7eb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:33.782323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:33.785590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 278/558] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:33.788340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e7d1f-f0a5-439d-9b4d-ef31cd8c0e5f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:33.792774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e7d1f-f0a5-439d-9b4d-ef31cd8c0e5f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:33.792774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e7d1f-f0a5-439d-9b4d-ef31cd8c0e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:33.813539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-776e7d1f-f0a5-439d-9b4d-ef31cd8c0e5f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:33.814783 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-776e7d1f-f0a5-439d-9b4d-ef31cd8c0e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:33.822036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe17b4-ee2d-448c-919e-729bf212c687 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:33.901768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d1c50-2e2c-4447-b051-aebaf557e663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:33.902505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d1c50-2e2c-4447-b051-aebaf557e663 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:33.902780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d1c50-2e2c-4447-b051-aebaf557e663 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:33.904627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d1c50-2e2c-4447-b051-aebaf557e663 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:33.909970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d1c50-2e2c-4447-b051-aebaf557e663 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:33.910287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d1c50-2e2c-4447-b051-aebaf557e663 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:33.911814 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d1c50-2e2c-4447-b051-aebaf557e663 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:33.916704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 279/559] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:33.918571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e7d1f-f0a5-439d-9b4d-ef31cd8c0e5f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:33.950468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 281/560] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Aug 29 23:30:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:33.950468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d1c50-2e2c-4447-b051-aebaf557e663 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:33.969707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:34.172562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d1c50-2e2c-4447-b051-aebaf557e663 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:34.197571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:34.216254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e7d1f-f0a5-439d-9b4d-ef31cd8c0e5f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:34.221985 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e20f3c-8ed2-4dcd-bb4d-e841819b81f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:34.224092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e20f3c-8ed2-4dcd-bb4d-e841819b81f0 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:34.224092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e20f3c-8ed2-4dcd-bb4d-e841819b81f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:34.224092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e20f3c-8ed2-4dcd-bb4d-e841819b81f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:34.224543 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e20f3c-8ed2-4dcd-bb4d-e841819b81f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:34.224914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e20f3c-8ed2-4dcd-bb4d-e841819b81f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:34.226377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e20f3c-8ed2-4dcd-bb4d-e841819b81f0 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:34.237099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 282/561] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:34.239588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e20f3c-8ed2-4dcd-bb4d-e841819b81f0 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:34.278390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7e416-9e14-47e4-90f2-c590c1b045ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:34.279391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7e416-9e14-47e4-90f2-c590c1b045ed None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:34.279841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7e416-9e14-47e4-90f2-c590c1b045ed None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:34.280286 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7e416-9e14-47e4-90f2-c590c1b045ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:34.294638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7e416-9e14-47e4-90f2-c590c1b045ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:34.299444 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7e416-9e14-47e4-90f2-c590c1b045ed None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a6a7e416-9e14-47e4-90f2-c590c1b045ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:34.313736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:34.314801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:34.315796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:34.315796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:34.315796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:34.315796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:34.315796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:34.315796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:34.315796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:34.315796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:34.315796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:30:34.315796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:34.317256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 280/562] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Aug 29 23:30:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:34.318210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7e416-9e14-47e4-90f2-c590c1b045ed None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:34.369800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:34.426492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e20f3c-8ed2-4dcd-bb4d-e841819b81f0 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:34.457981 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7e416-9e14-47e4-90f2-c590c1b045ed None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:34.482818 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c756ee-c594-4176-a8db-65453c937641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:34.483816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c756ee-c594-4176-a8db-65453c937641 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:34.484264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c756ee-c594-4176-a8db-65453c937641 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:34.484781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c756ee-c594-4176-a8db-65453c937641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:34.500579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582bdb81-a2c9-4d9e-a472-77884ef755f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:34.501231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bdb81-a2c9-4d9e-a472-77884ef755f9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:34.501505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bdb81-a2c9-4d9e-a472-77884ef755f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:34.501772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bdb81-a2c9-4d9e-a472-77884ef755f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:34.514901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582bdb81-a2c9-4d9e-a472-77884ef755f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:34.516427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582bdb81-a2c9-4d9e-a472-77884ef755f9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:34.526259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-582bdb81-a2c9-4d9e-a472-77884ef755f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:34.533866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:34.536498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:34.537400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:34.537400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:34.537400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:34.537400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 281/563] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:34.537400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2586159-0264-42b7-8075-355f07d3c1a4 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:34.537400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2586159-0264-42b7-8075-355f07d3c1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:34.537740 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2586159-0264-42b7-8075-355f07d3c1a4 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:34.539214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 282/564] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:34.539977 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2586159-0264-42b7-8075-355f07d3c1a4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:34.615907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c756ee-c594-4176-a8db-65453c937641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:34.618029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c756ee-c594-4176-a8db-65453c937641 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:34.684690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2586159-0264-42b7-8075-355f07d3c1a4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:34.714207 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:34.741218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadb28f-e47a-4fcf-a90b-66855aa30e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:34.742238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadb28f-e47a-4fcf-a90b-66855aa30e1d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:34.743098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadb28f-e47a-4fcf-a90b-66855aa30e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:34.743596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadb28f-e47a-4fcf-a90b-66855aa30e1d None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:34.745172 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadb28f-e47a-4fcf-a90b-66855aa30e1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:34.746350 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadb28f-e47a-4fcf-a90b-66855aa30e1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:34.748901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadb28f-e47a-4fcf-a90b-66855aa30e1d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:34.762502 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 283/565] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:34.763427 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c756ee-c594-4176-a8db-65453c937641 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:34.785679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 283/566] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Aug 29 23:30:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:34.786837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadb28f-e47a-4fcf-a90b-66855aa30e1d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:34.866605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:34.920229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c756ee-c594-4176-a8db-65453c937641 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:34.928775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadb28f-e47a-4fcf-a90b-66855aa30e1d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:34.976307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfde4e-7b16-4a73-9762-f0c9d66e5b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:34.977305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfde4e-7b16-4a73-9762-f0c9d66e5b0f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:34.977789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfde4e-7b16-4a73-9762-f0c9d66e5b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:34.978354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfde4e-7b16-4a73-9762-f0c9d66e5b0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:34.983886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9a0ca-e178-4f5a-85a0-af3241a0464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:34.984870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9a0ca-e178-4f5a-85a0-af3241a0464c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:34.985335 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9a0ca-e178-4f5a-85a0-af3241a0464c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:34.985777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9a0ca-e178-4f5a-85a0-af3241a0464c None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:34.986566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9a0ca-e178-4f5a-85a0-af3241a0464c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:34.987014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9a0ca-e178-4f5a-85a0-af3241a0464c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:34.989286 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9a0ca-e178-4f5a-85a0-af3241a0464c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:34.997617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfde4e-7b16-4a73-9762-f0c9d66e5b0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-4fdfde4e-7b16-4a73-9762-f0c9d66e5b0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:35.007364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:35.008361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:35.009196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:35.009839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 284/567] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:35.009973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebcf79-bedf-4684-b63a-228447bd35ba None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:35.010339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebcf79-bedf-4684-b63a-228447bd35ba None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:35.010775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebcf79-bedf-4684-b63a-228447bd35ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:35.014785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 284/568] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:35.014785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfde4e-7b16-4a73-9762-f0c9d66e5b0f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:35.040519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-69ebcf79-bedf-4684-b63a-228447bd35ba None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:35.041886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-69ebcf79-bedf-4684-b63a-228447bd35ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:35.059795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:35.147081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 285/569] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:35.148089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebcf79-bedf-4684-b63a-228447bd35ba None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:35.183520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfde4e-7b16-4a73-9762-f0c9d66e5b0f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:35.231423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95674ff6-1eca-42c7-9f75-cfbd0c4419f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:35.232050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95674ff6-1eca-42c7-9f75-cfbd0c4419f4 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:35.232365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95674ff6-1eca-42c7-9f75-cfbd0c4419f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:35.232603 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95674ff6-1eca-42c7-9f75-cfbd0c4419f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:35.246252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:35.257509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95674ff6-1eca-42c7-9f75-cfbd0c4419f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:35.260260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95674ff6-1eca-42c7-9f75-cfbd0c4419f4 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-95674ff6-1eca-42c7-9f75-cfbd0c4419f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:35.282105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:35.283209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:35.285128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:35.285128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:35.285128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:35.287508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 285/570] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:35.288678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95674ff6-1eca-42c7-9f75-cfbd0c4419f4 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:35.353266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebcf79-bedf-4684-b63a-228447bd35ba None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:35.432276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:35.434615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3583893-f0e4-45f7-8aa5-e4594214c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:35.435627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3583893-f0e4-45f7-8aa5-e4594214c63c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:35.435793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3583893-f0e4-45f7-8aa5-e4594214c63c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:35.436096 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3583893-f0e4-45f7-8aa5-e4594214c63c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:35.448687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3583893-f0e4-45f7-8aa5-e4594214c63c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:35.450559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3583893-f0e4-45f7-8aa5-e4594214c63c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c3583893-f0e4-45f7-8aa5-e4594214c63c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:35.470438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:35.472265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:35.474094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:35.474094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:35.474094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:35.474281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 286/571] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:35.476804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3583893-f0e4-45f7-8aa5-e4594214c63c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:35.482941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95674ff6-1eca-42c7-9f75-cfbd0c4419f4 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:35.549336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f7045-366d-415e-b41d-4f4a936964ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:35.551064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f7045-366d-415e-b41d-4f4a936964ec None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:35.551571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f7045-366d-415e-b41d-4f4a936964ec None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:35.552046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f7045-366d-415e-b41d-4f4a936964ec None admin] PATH_INFO: `/v3/users/glance` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:35.566644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f7045-366d-415e-b41d-4f4a936964ec None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:35.568072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f7045-366d-415e-b41d-4f4a936964ec None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-478f7045-366d-415e-b41d-4f4a936964ec None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:35.588529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:35.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:35.591678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:35.591678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:35.591678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:35.591678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:35.591678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:35.591678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:35.591678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:35.591678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:35.591678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 23:30:35.591678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:35.599301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 286/572] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:35.600788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f7045-366d-415e-b41d-4f4a936964ec None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:35.643027 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:35.647764 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3583893-f0e4-45f7-8aa5-e4594214c63c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:35.695770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed44882-f1cd-4a91-b29a-b5dd9bb8d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:35.696698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed44882-f1cd-4a91-b29a-b5dd9bb8d6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:35.697124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed44882-f1cd-4a91-b29a-b5dd9bb8d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:35.697674 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed44882-f1cd-4a91-b29a-b5dd9bb8d6b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:35.698739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed44882-f1cd-4a91-b29a-b5dd9bb8d6b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:35.699198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed44882-f1cd-4a91-b29a-b5dd9bb8d6b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:35.701115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed44882-f1cd-4a91-b29a-b5dd9bb8d6b9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:35.718863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 287/573] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:35.718863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed44882-f1cd-4a91-b29a-b5dd9bb8d6b9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:35.776103 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f7045-366d-415e-b41d-4f4a936964ec None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:35.829808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39d612-9921-488b-bfa5-e30c7f7b95df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:35.830467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39d612-9921-488b-bfa5-e30c7f7b95df None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:35.830734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39d612-9921-488b-bfa5-e30c7f7b95df None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:35.831638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39d612-9921-488b-bfa5-e30c7f7b95df None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:35.832502 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39d612-9921-488b-bfa5-e30c7f7b95df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:35.832800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39d612-9921-488b-bfa5-e30c7f7b95df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:35.834218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39d612-9921-488b-bfa5-e30c7f7b95df None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:35.845738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:35.851018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 287/574] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:35.852286 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39d612-9921-488b-bfa5-e30c7f7b95df None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:35.866128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed44882-f1cd-4a91-b29a-b5dd9bb8d6b9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:35.928709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8771e2-a5ba-4b78-ac58-9c7e93c90c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:35.929640 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8771e2-a5ba-4b78-ac58-9c7e93c90c7a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:35.930099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8771e2-a5ba-4b78-ac58-9c7e93c90c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:35.930612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8771e2-a5ba-4b78-ac58-9c7e93c90c7a None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:35.931693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8771e2-a5ba-4b78-ac58-9c7e93c90c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:35.932124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8771e2-a5ba-4b78-ac58-9c7e93c90c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:35.933651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8771e2-a5ba-4b78-ac58-9c7e93c90c7a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:35.949674 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 288/575] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:35.951120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8771e2-a5ba-4b78-ac58-9c7e93c90c7a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:35.985807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:35.999571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39d612-9921-488b-bfa5-e30c7f7b95df None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:36.054227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a7847-92b5-4156-b22d-cdb230471bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:36.054829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a7847-92b5-4156-b22d-cdb230471bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:36.055119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a7847-92b5-4156-b22d-cdb230471bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:36.055393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a7847-92b5-4156-b22d-cdb230471bb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:36.067023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a7847-92b5-4156-b22d-cdb230471bb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-a68a7847-92b5-4156-b22d-cdb230471bb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:36.072069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:36.073034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:36.073914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 288/576] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:36.074651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a7847-92b5-4156-b22d-cdb230471bb5 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:36.090552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8771e2-a5ba-4b78-ac58-9c7e93c90c7a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:36.157410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcdf85-0230-4627-b9b3-f11a5dfce8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:36.158485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcdf85-0230-4627-b9b3-f11a5dfce8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:36.158715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcdf85-0230-4627-b9b3-f11a5dfce8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:36.159522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcdf85-0230-4627-b9b3-f11a5dfce8c1 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:36.160979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcdf85-0230-4627-b9b3-f11a5dfce8c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:36.160979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcdf85-0230-4627-b9b3-f11a5dfce8c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:36.162406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcdf85-0230-4627-b9b3-f11a5dfce8c1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:36.163553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:36.186763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 289/577] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Aug 29 23:30:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:36.187670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcdf85-0230-4627-b9b3-f11a5dfce8c1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:36.248872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a7847-92b5-4156-b22d-cdb230471bb5 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:36.303321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a766b71-5cce-4778-a7d8-de43691b1d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:36.304334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a766b71-5cce-4778-a7d8-de43691b1d78 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:36.304692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a766b71-5cce-4778-a7d8-de43691b1d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:36.305113 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a766b71-5cce-4778-a7d8-de43691b1d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:36.325594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a766b71-5cce-4778-a7d8-de43691b1d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:36.327703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a766b71-5cce-4778-a7d8-de43691b1d78 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:36.340432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-7a766b71-5cce-4778-a7d8-de43691b1d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:36.355781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:36.357576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:36.359227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:36.359227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:36.359227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:36.359227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 289/578] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:36.359227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a766b71-5cce-4778-a7d8-de43691b1d78 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:36.378730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcdf85-0230-4627-b9b3-f11a5dfce8c1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:36.438469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395b073-5a7d-4250-bbb4-715ac96cef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:36.439111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395b073-5a7d-4250-bbb4-715ac96cef6a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:36.439388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395b073-5a7d-4250-bbb4-715ac96cef6a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:36.439785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395b073-5a7d-4250-bbb4-715ac96cef6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:36.448639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395b073-5a7d-4250-bbb4-715ac96cef6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:36.450138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395b073-5a7d-4250-bbb4-715ac96cef6a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d395b073-5a7d-4250-bbb4-715ac96cef6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:36.463872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:36.465498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:36.467074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:36.467074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:36.467074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:36.467074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:36.467074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:36.467074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:36.467074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:36.467074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:36.470002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 290/579] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:36.471006 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395b073-5a7d-4250-bbb4-715ac96cef6a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:36.524751 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a766b71-5cce-4778-a7d8-de43691b1d78 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:36.554687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:36.592107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de1218-f53d-495b-a8c5-b46e15f9d6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:36.593458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de1218-f53d-495b-a8c5-b46e15f9d6bb None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:36.594105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de1218-f53d-495b-a8c5-b46e15f9d6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:36.594805 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de1218-f53d-495b-a8c5-b46e15f9d6bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:36.608044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de1218-f53d-495b-a8c5-b46e15f9d6bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:36.610708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de1218-f53d-495b-a8c5-b46e15f9d6bb None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-32de1218-f53d-495b-a8c5-b46e15f9d6bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:36.631828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:36.632784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:36.633710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:36.633710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:36.633710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:36.633710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:36.633710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:36.633710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:36.633710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:36.633710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:36.638686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 290/580] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:36.640289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de1218-f53d-495b-a8c5-b46e15f9d6bb None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:36.666703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395b073-5a7d-4250-bbb4-715ac96cef6a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:36.723545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b01f-cccd-47ae-88d0-85d1da073f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:36.724519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b01f-cccd-47ae-88d0-85d1da073f89 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:36.724940 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b01f-cccd-47ae-88d0-85d1da073f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:36.725384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b01f-cccd-47ae-88d0-85d1da073f89 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:36.726533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455b01f-cccd-47ae-88d0-85d1da073f89 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:36.727025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455b01f-cccd-47ae-88d0-85d1da073f89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:36.728534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455b01f-cccd-47ae-88d0-85d1da073f89 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:36.743228 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 291/581] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:36.744873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b01f-cccd-47ae-88d0-85d1da073f89 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:36.749478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:36.819569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de1218-f53d-495b-a8c5-b46e15f9d6bb None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:36.873252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b01f-cccd-47ae-88d0-85d1da073f89 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:36.879104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f911e-a323-43d4-9943-f5b4494b0ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:36.879733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f911e-a323-43d4-9943-f5b4494b0ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:36.880010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f911e-a323-43d4-9943-f5b4494b0ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:36.880283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f911e-a323-43d4-9943-f5b4494b0ec9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:36.894339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f911e-a323-43d4-9943-f5b4494b0ec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:36.897427 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f911e-a323-43d4-9943-f5b4494b0ec9 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-728f911e-a323-43d4-9943-f5b4494b0ec9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:36.917485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:36.918546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:36.919442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:36.919442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:36.919442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:36.921342 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 291/582] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:36.922811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f911e-a323-43d4-9943-f5b4494b0ec9 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:36.933173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ae96e-5e2e-4660-a94d-ac2dd92bec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:36.933792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae96e-5e2e-4660-a94d-ac2dd92bec67 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:36.934070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae96e-5e2e-4660-a94d-ac2dd92bec67 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:36.934412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae96e-5e2e-4660-a94d-ac2dd92bec67 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:36.935040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ae96e-5e2e-4660-a94d-ac2dd92bec67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:36.935352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ae96e-5e2e-4660-a94d-ac2dd92bec67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:36.936802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ae96e-5e2e-4660-a94d-ac2dd92bec67 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:36.951310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 292/583] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:36.952296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ae96e-5e2e-4660-a94d-ac2dd92bec67 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:36.962718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:37.033691 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:37.065673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f911e-a323-43d4-9943-f5b4494b0ec9 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:37.109665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ae96e-5e2e-4660-a94d-ac2dd92bec67 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:37.115902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0115e60-471a-445f-8fa9-7b22b8f00e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:37.116531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0115e60-471a-445f-8fa9-7b22b8f00e88 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:37.116807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0115e60-471a-445f-8fa9-7b22b8f00e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:37.117069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0115e60-471a-445f-8fa9-7b22b8f00e88 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:37.117800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0115e60-471a-445f-8fa9-7b22b8f00e88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:37.118102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0115e60-471a-445f-8fa9-7b22b8f00e88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:37.120025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0115e60-471a-445f-8fa9-7b22b8f00e88 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:37.135022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 292/584] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:37.136024 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0115e60-471a-445f-8fa9-7b22b8f00e88 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:37.165606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80be919-98e8-48f5-a4a9-3e56bafd11c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:37.166629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be919-98e8-48f5-a4a9-3e56bafd11c5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:37.167083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be919-98e8-48f5-a4a9-3e56bafd11c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:37.167588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be919-98e8-48f5-a4a9-3e56bafd11c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:37.168527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80be919-98e8-48f5-a4a9-3e56bafd11c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:37.169056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80be919-98e8-48f5-a4a9-3e56bafd11c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:37.171381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80be919-98e8-48f5-a4a9-3e56bafd11c5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:37.189433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 293/585] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:37.191767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80be919-98e8-48f5-a4a9-3e56bafd11c5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:37.252859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:37.273790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0115e60-471a-445f-8fa9-7b22b8f00e88 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:37.316873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60957d-f68c-4600-9cbe-b64688dd48e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:37.317730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60957d-f68c-4600-9cbe-b64688dd48e7 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:37.318017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60957d-f68c-4600-9cbe-b64688dd48e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:37.318291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60957d-f68c-4600-9cbe-b64688dd48e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:37.331854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60957d-f68c-4600-9cbe-b64688dd48e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:37.333361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60957d-f68c-4600-9cbe-b64688dd48e7 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-de60957d-f68c-4600-9cbe-b64688dd48e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:37.356693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:37.357701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:37.359163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:37.359163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:37.359163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:37.359341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 293/586] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:37.361095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60957d-f68c-4600-9cbe-b64688dd48e7 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:37.395546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80be919-98e8-48f5-a4a9-3e56bafd11c5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:37.428998 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:37.450835 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa65bf4-a834-42dc-bf79-d5e0970d1531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:37.451495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa65bf4-a834-42dc-bf79-d5e0970d1531 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:37.451790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa65bf4-a834-42dc-bf79-d5e0970d1531 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:37.452164 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa65bf4-a834-42dc-bf79-d5e0970d1531 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:37.452852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa65bf4-a834-42dc-bf79-d5e0970d1531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:37.453250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa65bf4-a834-42dc-bf79-d5e0970d1531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:37.456211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa65bf4-a834-42dc-bf79-d5e0970d1531 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:37.475016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 294/587] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:37.475905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa65bf4-a834-42dc-bf79-d5e0970d1531 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:37.530837 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60957d-f68c-4600-9cbe-b64688dd48e7 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:37.578022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1c9f2-16e8-43dd-905f-ea426022f897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:37.578819 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1c9f2-16e8-43dd-905f-ea426022f897 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:37.579099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1c9f2-16e8-43dd-905f-ea426022f897 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:37.579369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1c9f2-16e8-43dd-905f-ea426022f897 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:37.582751 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:37.589319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1c9f2-16e8-43dd-905f-ea426022f897 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:37.595828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1c9f2-16e8-43dd-905f-ea426022f897 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-a3c1c9f2-16e8-43dd-905f-ea426022f897 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:37.605484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:37.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:37.607786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:37.607786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:37.607786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:37.607786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:37.607786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:37.607786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:37.607786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:37.607786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:37.607786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 23:30:37.607786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:37.607786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 294/588] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:37.608717 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1c9f2-16e8-43dd-905f-ea426022f897 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:37.635644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa65bf4-a834-42dc-bf79-d5e0970d1531 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:37.676306 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55951b8-a9d4-440c-8d44-0733ee8b8712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:37.676952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55951b8-a9d4-440c-8d44-0733ee8b8712 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:37.677232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55951b8-a9d4-440c-8d44-0733ee8b8712 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:37.677504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55951b8-a9d4-440c-8d44-0733ee8b8712 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:37.678252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55951b8-a9d4-440c-8d44-0733ee8b8712 None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=c6b37b09a1ae45e8bc16adfa59500eb9) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:37.678604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55951b8-a9d4-440c-8d44-0733ee8b8712 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:37.680073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55951b8-a9d4-440c-8d44-0733ee8b8712 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:37.687121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 295/589] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:30:37 2022] GET /identity/v3/role_assignments?user.id=c6b37b09a1ae45e8bc16adfa59500eb9&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:37.688012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55951b8-a9d4-440c-8d44-0733ee8b8712 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:37.752001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:37.782648 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1c9f2-16e8-43dd-905f-ea426022f897 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:37.843159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735e2f2-3ff2-4657-b38c-f461115d9e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:37.844280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735e2f2-3ff2-4657-b38c-f461115d9e57 None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:37.844280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735e2f2-3ff2-4657-b38c-f461115d9e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:37.844595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735e2f2-3ff2-4657-b38c-f461115d9e57 None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/e4a8261d92064b838fc6e8f415879566/roles/f0926be4d3d4462fac854212e101ccdd` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:37.860321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55951b8-a9d4-440c-8d44-0733ee8b8712 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:37.866117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735e2f2-3ff2-4657-b38c-f461115d9e57 None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=e4a8261d92064b838fc6e8f415879566, role_id=f0926be4d3d4462fac854212e101ccdd)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:37.867588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735e2f2-3ff2-4657-b38c-f461115d9e57 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:37.910593 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5f0aa-fdd4-4913-b908-6981ccd1c93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:37.911320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5f0aa-fdd4-4913-b908-6981ccd1c93b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:37.912435 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5f0aa-fdd4-4913-b908-6981ccd1c93b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:37.912739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5f0aa-fdd4-4913-b908-6981ccd1c93b None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:37.913554 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5f0aa-fdd4-4913-b908-6981ccd1c93b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:37.913858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5f0aa-fdd4-4913-b908-6981ccd1c93b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:37.915371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5f0aa-fdd4-4913-b908-6981ccd1c93b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:37.931463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 296/590] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:37.932522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5f0aa-fdd4-4913-b908-6981ccd1c93b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:38.088297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5f0aa-fdd4-4913-b908-6981ccd1c93b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:38.100544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:38.135629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670ccab-334b-4686-9403-5c1693b7ef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:38.136604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670ccab-334b-4686-9403-5c1693b7ef63 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:38.136914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670ccab-334b-4686-9403-5c1693b7ef63 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:38.137148 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670ccab-334b-4686-9403-5c1693b7ef63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:38.156904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670ccab-334b-4686-9403-5c1693b7ef63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:38.160487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670ccab-334b-4686-9403-5c1693b7ef63 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-4670ccab-334b-4686-9403-5c1693b7ef63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:38.176579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:38.178347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:38.180168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:38.180168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:38.180168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:38.180168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:38.180168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:38.180168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:38.180168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:38.180168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:38.185442 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 297/591] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:38.187085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670ccab-334b-4686-9403-5c1693b7ef63 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:38.266779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 295/592] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:30:37 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/e4a8261d92064b838fc6e8f415879566/roles/f0926be4d3d4462fac854212e101ccdd => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:30:38.268087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735e2f2-3ff2-4657-b38c-f461115d9e57 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:38.390369 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670ccab-334b-4686-9403-5c1693b7ef63 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:38.457016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b9011-dde5-43d4-9ff3-96bc17e4443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:38.457850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:38.458249 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b9011-dde5-43d4-9ff3-96bc17e4443e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:38.458759 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b9011-dde5-43d4-9ff3-96bc17e4443e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:38.459253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b9011-dde5-43d4-9ff3-96bc17e4443e None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:38.461805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b9011-dde5-43d4-9ff3-96bc17e4443e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:38.462415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b9011-dde5-43d4-9ff3-96bc17e4443e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:38.465010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b9011-dde5-43d4-9ff3-96bc17e4443e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:38.493291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 298/593] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Aug 29 23:30:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:38.496246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b9011-dde5-43d4-9ff3-96bc17e4443e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:38.536504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:38.577398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735e2f2-3ff2-4657-b38c-f461115d9e57 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:38.654455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d3cb4-0b60-48f8-b7b4-0283dd96affd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:38.654455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d3cb4-0b60-48f8-b7b4-0283dd96affd None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:38.654455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d3cb4-0b60-48f8-b7b4-0283dd96affd None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:38.654455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d3cb4-0b60-48f8-b7b4-0283dd96affd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:38.673981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d3cb4-0b60-48f8-b7b4-0283dd96affd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:38.673981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d3cb4-0b60-48f8-b7b4-0283dd96affd None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-c47d3cb4-0b60-48f8-b7b4-0283dd96affd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:38.698446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:38.699463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:38.700393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:38.700393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:38.700393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:38.700393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:38.700393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:38.700393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:38.700393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:38.700393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:38.700393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 23:30:38.700393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:38.700945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b9011-dde5-43d4-9ff3-96bc17e4443e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:38.710528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 296/594] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:38.710528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d3cb4-0b60-48f8-b7b4-0283dd96affd None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:38.766451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccc99b-5e3b-4abb-9332-0093ac7c9d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:38.766451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccc99b-5e3b-4abb-9332-0093ac7c9d48 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:38.766451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccc99b-5e3b-4abb-9332-0093ac7c9d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:38.766451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccc99b-5e3b-4abb-9332-0093ac7c9d48 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:38.766908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccc99b-5e3b-4abb-9332-0093ac7c9d48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:38.766908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccc99b-5e3b-4abb-9332-0093ac7c9d48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:38.770444 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccc99b-5e3b-4abb-9332-0093ac7c9d48 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:38.806603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 299/595] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:38.810446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccc99b-5e3b-4abb-9332-0093ac7c9d48 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:38.993161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d3cb4-0b60-48f8-b7b4-0283dd96affd None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:39.010463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:39.017201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:39.054449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccc99b-5e3b-4abb-9332-0093ac7c9d48 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:39.069013 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730075b-33a8-4ac9-a9e2-bc7e75d42c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:39.069981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730075b-33a8-4ac9-a9e2-bc7e75d42c8f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:39.070329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730075b-33a8-4ac9-a9e2-bc7e75d42c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:39.070896 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730075b-33a8-4ac9-a9e2-bc7e75d42c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:39.092360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730075b-33a8-4ac9-a9e2-bc7e75d42c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:39.098503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730075b-33a8-4ac9-a9e2-bc7e75d42c8f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-4730075b-33a8-4ac9-a9e2-bc7e75d42c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:39.126760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:39.128561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:39.129530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:39.129530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:39.129530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:39.134466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 297/596] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:39.134466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730075b-33a8-4ac9-a9e2-bc7e75d42c8f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:39.146461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c58158-7602-4239-9dba-77df1861fd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:39.146461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c58158-7602-4239-9dba-77df1861fd66 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:39.146461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c58158-7602-4239-9dba-77df1861fd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:39.146461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c58158-7602-4239-9dba-77df1861fd66 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:39.149272 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c58158-7602-4239-9dba-77df1861fd66 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:39.149812 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c58158-7602-4239-9dba-77df1861fd66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:39.155652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c58158-7602-4239-9dba-77df1861fd66 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:39.202119 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 300/597] 10.222.0.58 () {56 vars in 1139 bytes} [Mon Aug 29 23:30:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:39.204745 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c58158-7602-4239-9dba-77df1861fd66 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:39.294223 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730075b-33a8-4ac9-a9e2-bc7e75d42c8f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:39.342450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f8a67-1f65-4887-aac2-aaf6774840bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:39.342450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f8a67-1f65-4887-aac2-aaf6774840bf None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:39.342450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f8a67-1f65-4887-aac2-aaf6774840bf None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:39.342803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f8a67-1f65-4887-aac2-aaf6774840bf None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/d7a59337d20e4678b8a16ac35bffbdb4/roles/f0926be4d3d4462fac854212e101ccdd` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:39.368450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f8a67-1f65-4887-aac2-aaf6774840bf None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=d7a59337d20e4678b8a16ac35bffbdb4, role_id=f0926be4d3d4462fac854212e101ccdd)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:39.371370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f8a67-1f65-4887-aac2-aaf6774840bf None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:39.398851 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c58158-7602-4239-9dba-77df1861fd66 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:39.441586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fefcc-108b-4f7b-8b27-08c0a1fb4d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:39.442212 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fefcc-108b-4f7b-8b27-08c0a1fb4d56 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:39.444019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fefcc-108b-4f7b-8b27-08c0a1fb4d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:39.444019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fefcc-108b-4f7b-8b27-08c0a1fb4d56 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:39.444019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439fefcc-108b-4f7b-8b27-08c0a1fb4d56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:39.444453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439fefcc-108b-4f7b-8b27-08c0a1fb4d56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:39.445662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439fefcc-108b-4f7b-8b27-08c0a1fb4d56 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:39.464149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 301/598] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:39.464149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fefcc-108b-4f7b-8b27-08c0a1fb4d56 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:39.464434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 298/599] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:30:39 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/d7a59337d20e4678b8a16ac35bffbdb4/roles/f0926be4d3d4462fac854212e101ccdd => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:30:39.470588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f8a67-1f65-4887-aac2-aaf6774840bf None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:39.475943 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:39.498019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:39.743443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fefcc-108b-4f7b-8b27-08c0a1fb4d56 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:39.798456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f8a67-1f65-4887-aac2-aaf6774840bf None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:39.855237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938646b-0adc-4cf0-911a-9d97b018b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:39.856391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938646b-0adc-4cf0-911a-9d97b018b14d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:39.856903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938646b-0adc-4cf0-911a-9d97b018b14d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:39.857381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938646b-0adc-4cf0-911a-9d97b018b14d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:39.870545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790e1c3-eba4-42e1-a187-285e7351c095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:39.871567 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790e1c3-eba4-42e1-a187-285e7351c095 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:39.872043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790e1c3-eba4-42e1-a187-285e7351c095 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:39.872533 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790e1c3-eba4-42e1-a187-285e7351c095 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:39.892395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790e1c3-eba4-42e1-a187-285e7351c095 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:39.895831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790e1c3-eba4-42e1-a187-285e7351c095 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:39.897455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938646b-0adc-4cf0-911a-9d97b018b14d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:39.900325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938646b-0adc-4cf0-911a-9d97b018b14d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-5790e1c3-eba4-42e1-a187-285e7351c095 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:39.924075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:39.925161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:39.926284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:39.926284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:39.926284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:39.926284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:39.926284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:39.926284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:39.926284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:39.926284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:39.927229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 299/600] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:39.930167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1708bf-310c-416a-86ee-d75c79a7264d None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:39.930167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1708bf-310c-416a-86ee-d75c79a7264d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:39.930167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1708bf-310c-416a-86ee-d75c79a7264d None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:39.942075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 300/601] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:39 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:39.942720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1708bf-310c-416a-86ee-d75c79a7264d None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-7938646b-0adc-4cf0-911a-9d97b018b14d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:39.950097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:39.951136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:39.952042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:39.952042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:39.952042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:39.956308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 302/602] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:39.957975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0ee2b-cf85-4445-9be7-633db91061fb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:39.958572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0ee2b-cf85-4445-9be7-633db91061fb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:39.959051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0ee2b-cf85-4445-9be7-633db91061fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:40.024129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c2b0ee2b-cf85-4445-9be7-633db91061fb None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:40.025042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c2b0ee2b-cf85-4445-9be7-633db91061fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:40.199253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:40.213162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:40.272932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 303/603] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:40.272932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0ee2b-cf85-4445-9be7-633db91061fb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:40.276194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1708bf-310c-416a-86ee-d75c79a7264d None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:40.417706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df0936-a2cc-4f42-ac7d-b7ef5fca8b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:40.423013 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df0936-a2cc-4f42-ac7d-b7ef5fca8b6f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:40.423470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df0936-a2cc-4f42-ac7d-b7ef5fca8b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:40.423874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df0936-a2cc-4f42-ac7d-b7ef5fca8b6f None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:40.426319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df0936-a2cc-4f42-ac7d-b7ef5fca8b6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:40.433212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df0936-a2cc-4f42-ac7d-b7ef5fca8b6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:40.435804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df0936-a2cc-4f42-ac7d-b7ef5fca8b6f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:40.497075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 301/604] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 555 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:40.526172 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df0936-a2cc-4f42-ac7d-b7ef5fca8b6f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:40.603494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0ee2b-cf85-4445-9be7-633db91061fb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:40.685093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d4034-9601-4ee7-923c-70f0ae294a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:40.685878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d4034-9601-4ee7-923c-70f0ae294a24 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:40.686154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d4034-9601-4ee7-923c-70f0ae294a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:40.686689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d4034-9601-4ee7-923c-70f0ae294a24 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:40.688318 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d4034-9601-4ee7-923c-70f0ae294a24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:40.688640 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d4034-9601-4ee7-923c-70f0ae294a24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:40.690855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d4034-9601-4ee7-923c-70f0ae294a24 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:40.716767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 304/605] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:40.722457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d4034-9601-4ee7-923c-70f0ae294a24 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:40.760121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df0936-a2cc-4f42-ac7d-b7ef5fca8b6f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:40.784182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:40.818545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:40.852576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf170b-4c55-4c29-9d46-2bfd5d48e7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:40.853421 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf170b-4c55-4c29-9d46-2bfd5d48e7bd None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:40.856874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf170b-4c55-4c29-9d46-2bfd5d48e7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:40.856874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf170b-4c55-4c29-9d46-2bfd5d48e7bd None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/64a42591a6814a0cbcb8444829cb25b3/roles/f0926be4d3d4462fac854212e101ccdd` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:40.861442 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d4034-9601-4ee7-923c-70f0ae294a24 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:40.887210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf170b-4c55-4c29-9d46-2bfd5d48e7bd None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=64a42591a6814a0cbcb8444829cb25b3, role_id=f0926be4d3d4462fac854212e101ccdd)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:40.888671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf170b-4c55-4c29-9d46-2bfd5d48e7bd None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:40.928953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fbf16-54b3-4ccb-8ae0-a5409c8531d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:40.929628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fbf16-54b3-4ccb-8ae0-a5409c8531d3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:40.930024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fbf16-54b3-4ccb-8ae0-a5409c8531d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:40.930301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fbf16-54b3-4ccb-8ae0-a5409c8531d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:40.944413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fbf16-54b3-4ccb-8ae0-a5409c8531d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-025fbf16-54b3-4ccb-8ae0-a5409c8531d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:40.949953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:40.956167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 305/606] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:40.957413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fbf16-54b3-4ccb-8ae0-a5409c8531d3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:40.974365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 302/607] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:30:40 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/64a42591a6814a0cbcb8444829cb25b3/roles/f0926be4d3d4462fac854212e101ccdd => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:30:40.978808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d2f35-df10-4ca7-9ee7-edd626c35008 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:40.978808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d2f35-df10-4ca7-9ee7-edd626c35008 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:40.978808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d2f35-df10-4ca7-9ee7-edd626c35008 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:40.978808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 303/608] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:40.978808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d2f35-df10-4ca7-9ee7-edd626c35008 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:41.131622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fbf16-54b3-4ccb-8ae0-a5409c8531d3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:41.141761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:41.190797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e5c8b-a348-4b78-ba94-65c4ea1666a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:41.193452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5c8b-a348-4b78-ba94-65c4ea1666a9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:41.193737 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5c8b-a348-4b78-ba94-65c4ea1666a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:41.194017 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5c8b-a348-4b78-ba94-65c4ea1666a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:41.204199 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e5c8b-a348-4b78-ba94-65c4ea1666a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:41.207725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e5c8b-a348-4b78-ba94-65c4ea1666a9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-644e5c8b-a348-4b78-ba94-65c4ea1666a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:41.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:41.225659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:41.227313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:41.227313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:41.227313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:41.227313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:41.227313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:41.227313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:41.227313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:41.227313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:41.227313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 306/609] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:41.230135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b850b-39e8-4c62-850c-a3d36748cd58 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:41.230558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b850b-39e8-4c62-850c-a3d36748cd58 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:41.230744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b850b-39e8-4c62-850c-a3d36748cd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:41.250398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9e2b850b-39e8-4c62-850c-a3d36748cd58 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:41.251781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9e2b850b-39e8-4c62-850c-a3d36748cd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:41.258472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d2f35-df10-4ca7-9ee7-edd626c35008 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:41.308187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fa717-28ac-4bc9-96f0-bb809e1dd627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:41.309443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fa717-28ac-4bc9-96f0-bb809e1dd627 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:41.309951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fa717-28ac-4bc9-96f0-bb809e1dd627 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:41.310559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fa717-28ac-4bc9-96f0-bb809e1dd627 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:41.311482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8fa717-28ac-4bc9-96f0-bb809e1dd627 None admin] RBAC: Adding query filter params (name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:41.314477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8fa717-28ac-4bc9-96f0-bb809e1dd627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:41.314477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8fa717-28ac-4bc9-96f0-bb809e1dd627 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:41.326227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 304/610] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:41.327735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fa717-28ac-4bc9-96f0-bb809e1dd627 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:41.395023 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 307/611] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:41.396307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b850b-39e8-4c62-850c-a3d36748cd58 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:41.406869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:41.491856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fa717-28ac-4bc9-96f0-bb809e1dd627 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:41.536986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6256627-07f7-49f0-a820-827ec8785fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:41.537612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6256627-07f7-49f0-a820-827ec8785fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:41.537895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6256627-07f7-49f0-a820-827ec8785fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:41.538169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6256627-07f7-49f0-a820-827ec8785fa1 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:41.538928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6256627-07f7-49f0-a820-827ec8785fa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:41.539237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6256627-07f7-49f0-a820-827ec8785fa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:41.540952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6256627-07f7-49f0-a820-827ec8785fa1 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:41.555567 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 305/612] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:41.563681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6256627-07f7-49f0-a820-827ec8785fa1 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:41.741609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:41.771320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b850b-39e8-4c62-850c-a3d36748cd58 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:41.781933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:41.785744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6256627-07f7-49f0-a820-827ec8785fa1 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:41.825961 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56b882-c8db-476e-94ff-9a1057fa4b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:41.826907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56b882-c8db-476e-94ff-9a1057fa4b88 None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:41.827362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56b882-c8db-476e-94ff-9a1057fa4b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:41.827772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56b882-c8db-476e-94ff-9a1057fa4b88 None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/863bcfa6b8694182b9d163133a141907/roles/f0926be4d3d4462fac854212e101ccdd` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:41.842436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7809b-cb2e-48b4-ad51-bd3af400a7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:41.843644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7809b-cb2e-48b4-ad51-bd3af400a7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:41.843644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7809b-cb2e-48b4-ad51-bd3af400a7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:41.844166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7809b-cb2e-48b4-ad51-bd3af400a7a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:41.847545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56b882-c8db-476e-94ff-9a1057fa4b88 None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=863bcfa6b8694182b9d163133a141907, role_id=f0926be4d3d4462fac854212e101ccdd)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:41.849209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56b882-c8db-476e-94ff-9a1057fa4b88 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:41.866267 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7809b-cb2e-48b4-ad51-bd3af400a7a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:41.868216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7809b-cb2e-48b4-ad51-bd3af400a7a5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1ed7809b-cb2e-48b4-ad51-bd3af400a7a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:41.895832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:41.896949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:41.898039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:41.898039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:41.898039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:41.898039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 308/613] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 501 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:41.898304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7809b-cb2e-48b4-ad51-bd3af400a7a5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:41.960364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 306/614] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:30:41 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/863bcfa6b8694182b9d163133a141907/roles/f0926be4d3d4462fac854212e101ccdd => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:30:41.962344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56b882-c8db-476e-94ff-9a1057fa4b88 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:42.105947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7809b-cb2e-48b4-ad51-bd3af400a7a5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:42.155003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ddfd2-54d2-4be0-a43c-d3224b05a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:42.155653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ddfd2-54d2-4be0-a43c-d3224b05a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:42.155917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ddfd2-54d2-4be0-a43c-d3224b05a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:42.156182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ddfd2-54d2-4be0-a43c-d3224b05a5c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:42.179432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ddfd2-54d2-4be0-a43c-d3224b05a5c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-b34ddfd2-54d2-4be0-a43c-d3224b05a5c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:42.185364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:42.186366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:42.187291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 309/615] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:42.188508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fcd0f-5000-46f2-8a13-057733ae81c1 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:42.188508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fcd0f-5000-46f2-8a13-057733ae81c1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:42.188749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fcd0f-5000-46f2-8a13-057733ae81c1 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:42.191930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 310/616] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:42.200332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e135f55-a7b3-49df-b622-b9174b79c02e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:42.200746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e135f55-a7b3-49df-b622-b9174b79c02e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:42.201038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e135f55-a7b3-49df-b622-b9174b79c02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:42.220364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56b882-c8db-476e-94ff-9a1057fa4b88 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:42.227204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:42.234155 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3e135f55-a7b3-49df-b622-b9174b79c02e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:42.236158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3e135f55-a7b3-49df-b622-b9174b79c02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:42.277947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc985b57-5a38-4a80-8a0f-fa812e403926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:42.278639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc985b57-5a38-4a80-8a0f-fa812e403926 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:42.278922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc985b57-5a38-4a80-8a0f-fa812e403926 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:42.279196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc985b57-5a38-4a80-8a0f-fa812e403926 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:42.279864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc985b57-5a38-4a80-8a0f-fa812e403926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:42.280168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc985b57-5a38-4a80-8a0f-fa812e403926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:42.281569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc985b57-5a38-4a80-8a0f-fa812e403926 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:42.295358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 307/617] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:42.296304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc985b57-5a38-4a80-8a0f-fa812e403926 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:42.312622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:42.388776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 311/618] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:42.388776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e135f55-a7b3-49df-b622-b9174b79c02e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:42.461088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc985b57-5a38-4a80-8a0f-fa812e403926 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:42.538266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208217de-4419-4f6d-9959-cce112874501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:42.539359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208217de-4419-4f6d-9959-cce112874501 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:42.539907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208217de-4419-4f6d-9959-cce112874501 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:42.540452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208217de-4419-4f6d-9959-cce112874501 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:42.541474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208217de-4419-4f6d-9959-cce112874501 None admin] RBAC: Adding query filter params (name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:42.542012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208217de-4419-4f6d-9959-cce112874501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:42.543677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208217de-4419-4f6d-9959-cce112874501 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:42.556473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 308/619] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:42.564970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208217de-4419-4f6d-9959-cce112874501 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:42.636984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e135f55-a7b3-49df-b622-b9174b79c02e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:42.728435 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338f4e0-af38-47ee-b68f-ec45098c66af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:42.729546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338f4e0-af38-47ee-b68f-ec45098c66af None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:42.730027 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338f4e0-af38-47ee-b68f-ec45098c66af None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:42.730752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338f4e0-af38-47ee-b68f-ec45098c66af None admin] PATH_INFO: `/v3/users/swift` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:42.735715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:42.746544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338f4e0-af38-47ee-b68f-ec45098c66af None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:42.761034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338f4e0-af38-47ee-b68f-ec45098c66af None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-0338f4e0-af38-47ee-b68f-ec45098c66af None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:42.764100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:42.765095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:42.766047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:42.766047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:42.766047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:42.766047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:42.766047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:42.766047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:42.766047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:42.766047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:42.766047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:30:42.766047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:42.769200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 312/620] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:30:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:42.781418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338f4e0-af38-47ee-b68f-ec45098c66af None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:42.783320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:42.822042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208217de-4419-4f6d-9959-cce112874501 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:42.863464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c483b2-76dd-4d96-8cb1-ad6532dad539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:42.864558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c483b2-76dd-4d96-8cb1-ad6532dad539 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:42.864558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c483b2-76dd-4d96-8cb1-ad6532dad539 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:42.864797 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c483b2-76dd-4d96-8cb1-ad6532dad539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:42.881902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c483b2-76dd-4d96-8cb1-ad6532dad539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:42.882387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c483b2-76dd-4d96-8cb1-ad6532dad539 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-c2c483b2-76dd-4d96-8cb1-ad6532dad539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:42.907691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:42.909294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:42.910188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:42.910188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:42.910188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:42.916419 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 309/621] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:42.917399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c483b2-76dd-4d96-8cb1-ad6532dad539 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:42.942394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338f4e0-af38-47ee-b68f-ec45098c66af None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:42.991670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa8a4e-a65f-445d-9a30-e15bbac8437b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:42.992462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8a4e-a65f-445d-9a30-e15bbac8437b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:42.992771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8a4e-a65f-445d-9a30-e15bbac8437b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:42.993141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8a4e-a65f-445d-9a30-e15bbac8437b None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:42.994344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa8a4e-a65f-445d-9a30-e15bbac8437b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:42.994795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa8a4e-a65f-445d-9a30-e15bbac8437b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:42.996560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa8a4e-a65f-445d-9a30-e15bbac8437b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:43.036120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 313/622] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Aug 29 23:30:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:43.040709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa8a4e-a65f-445d-9a30-e15bbac8437b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:43.051722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c483b2-76dd-4d96-8cb1-ad6532dad539 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:43.100333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce067c-1f00-47c2-b753-5be76558ea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:43.100987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce067c-1f00-47c2-b753-5be76558ea16 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:43.101244 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce067c-1f00-47c2-b753-5be76558ea16 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:43.105312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce067c-1f00-47c2-b753-5be76558ea16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:43.111424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:43.120903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce067c-1f00-47c2-b753-5be76558ea16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-f8ce067c-1f00-47c2-b753-5be76558ea16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:43.130409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:43.145885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 310/623] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:43.147938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce067c-1f00-47c2-b753-5be76558ea16 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:43.152693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:43.244881 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa8a4e-a65f-445d-9a30-e15bbac8437b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:43.273434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce067c-1f00-47c2-b753-5be76558ea16 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:43.316778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20dc19-318c-4b93-9f9a-ef9c99c2a4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:43.318220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20dc19-318c-4b93-9f9a-ef9c99c2a4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:43.318942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20dc19-318c-4b93-9f9a-ef9c99c2a4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:43.319426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20dc19-318c-4b93-9f9a-ef9c99c2a4d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:43.326405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18107dfa-f0ae-4bb4-a611-85e0af21e940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:43.328939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18107dfa-f0ae-4bb4-a611-85e0af21e940 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:43.329347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18107dfa-f0ae-4bb4-a611-85e0af21e940 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:43.329532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18107dfa-f0ae-4bb4-a611-85e0af21e940 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:43.331482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18107dfa-f0ae-4bb4-a611-85e0af21e940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:43.331482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18107dfa-f0ae-4bb4-a611-85e0af21e940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:43.332622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18107dfa-f0ae-4bb4-a611-85e0af21e940 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:43.336472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20dc19-318c-4b93-9f9a-ef9c99c2a4d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:43.338328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20dc19-318c-4b93-9f9a-ef9c99c2a4d4 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:43.357162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 314/624] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:43.357782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18107dfa-f0ae-4bb4-a611-85e0af21e940 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-bd20dc19-318c-4b93-9f9a-ef9c99c2a4d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:43.358494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:43.359452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:43.360446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:43.360446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:43.360446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:43.364976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 311/625] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:43.366295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20dc19-318c-4b93-9f9a-ef9c99c2a4d4 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:43.421541 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:43.529616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20dc19-318c-4b93-9f9a-ef9c99c2a4d4 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:43.563861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18107dfa-f0ae-4bb4-a611-85e0af21e940 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:43.583456 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:43.586003 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3ffa5-2a10-4e87-9226-a7460f643d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:43.586677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3ffa5-2a10-4e87-9226-a7460f643d69 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:43.586989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3ffa5-2a10-4e87-9226-a7460f643d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:43.587225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3ffa5-2a10-4e87-9226-a7460f643d69 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:43.602233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3ffa5-2a10-4e87-9226-a7460f643d69 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:43.606112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3ffa5-2a10-4e87-9226-a7460f643d69 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:43.614396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf182a-32e0-4995-97d3-c7dc633f4f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:43.615317 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf182a-32e0-4995-97d3-c7dc633f4f22 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:43.615787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf182a-32e0-4995-97d3-c7dc633f4f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:43.616222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf182a-32e0-4995-97d3-c7dc633f4f22 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:43.617100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf182a-32e0-4995-97d3-c7dc633f4f22 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:43.617586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf182a-32e0-4995-97d3-c7dc633f4f22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-86f3ffa5-2a10-4e87-9226-a7460f643d69 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:43.618234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:43.619229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:43.620169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:43.620169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:43.620169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:43.620169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:43.620169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:43.620169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:43.620169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:43.620169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:43.620169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:30:43.620169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:43.622359 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf182a-32e0-4995-97d3-c7dc633f4f22 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:43.626069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 312/626] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Aug 29 23:30:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:43.627983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3ffa5-2a10-4e87-9226-a7460f643d69 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:43.641429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 315/627] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:43.643581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61add21-35d0-4180-a399-7084bd700725 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:43.644541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61add21-35d0-4180-a399-7084bd700725 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:43.645009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61add21-35d0-4180-a399-7084bd700725 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:43.648110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 316/628] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:43.649591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61add21-35d0-4180-a399-7084bd700725 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:43.805332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:43.815231 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3ffa5-2a10-4e87-9226-a7460f643d69 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:43.824815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61add21-35d0-4180-a399-7084bd700725 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:43.858734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769c350-cf05-40ec-96ea-d142f87e33a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:43.859106 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769c350-cf05-40ec-96ea-d142f87e33a7 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:43.859507 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769c350-cf05-40ec-96ea-d142f87e33a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:43.859691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769c350-cf05-40ec-96ea-d142f87e33a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:43.860726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769c350-cf05-40ec-96ea-d142f87e33a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:43.860726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769c350-cf05-40ec-96ea-d142f87e33a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:43.861990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769c350-cf05-40ec-96ea-d142f87e33a7 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:43.877712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 313/629] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:43.878788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769c350-cf05-40ec-96ea-d142f87e33a7 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:43.879047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fa6e7-beed-423d-a41a-313ff5bbc5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:43.879677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fa6e7-beed-423d-a41a-313ff5bbc5ce None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:43.879959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fa6e7-beed-423d-a41a-313ff5bbc5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:43.880230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fa6e7-beed-423d-a41a-313ff5bbc5ce None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:43.882655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180fa6e7-beed-423d-a41a-313ff5bbc5ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:43.882655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180fa6e7-beed-423d-a41a-313ff5bbc5ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:43.883187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180fa6e7-beed-423d-a41a-313ff5bbc5ce None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:43.918044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 317/630] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Aug 29 23:30:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:43.918044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c001d-4c91-4600-aacf-005e9fee902a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:43.918044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c001d-4c91-4600-aacf-005e9fee902a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:43.918044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c001d-4c91-4600-aacf-005e9fee902a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:43.935948 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-401c001d-4c91-4600-aacf-005e9fee902a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:43.937178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-401c001d-4c91-4600-aacf-005e9fee902a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:43.992678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:44.052623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769c350-cf05-40ec-96ea-d142f87e33a7 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:44.073847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 318/631] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:44.075321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c001d-4c91-4600-aacf-005e9fee902a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:44.108616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd539cba-d84e-436c-a229-9d9f69a9ee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:44.109571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd539cba-d84e-436c-a229-9d9f69a9ee8f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:44.110028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd539cba-d84e-436c-a229-9d9f69a9ee8f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:44.110677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd539cba-d84e-436c-a229-9d9f69a9ee8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:44.135452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd539cba-d84e-436c-a229-9d9f69a9ee8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:44.136397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd539cba-d84e-436c-a229-9d9f69a9ee8f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-bd539cba-d84e-436c-a229-9d9f69a9ee8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:44.166989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:44.168002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:44.168922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:44.168922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:44.168922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:44.168922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 314/632] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:44.168922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd539cba-d84e-436c-a229-9d9f69a9ee8f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:44.202145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:44.288714 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c001d-4c91-4600-aacf-005e9fee902a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:44.354204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd539cba-d84e-436c-a229-9d9f69a9ee8f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:44.356165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774b048-c2b0-41e1-b659-462d7bf7634d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:44.357017 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b048-c2b0-41e1-b659-462d7bf7634d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:44.357460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b048-c2b0-41e1-b659-462d7bf7634d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:44.357740 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b048-c2b0-41e1-b659-462d7bf7634d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:44.368116 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774b048-c2b0-41e1-b659-462d7bf7634d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:44.369987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774b048-c2b0-41e1-b659-462d7bf7634d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e774b048-c2b0-41e1-b659-462d7bf7634d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:44.382991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:44.383966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:44.384885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:44.384885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:44.384885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:44.384885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:44.384885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:44.384885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:44.384885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:44.384885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:44.387416 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 319/633] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:44.388677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774b048-c2b0-41e1-b659-462d7bf7634d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:44.405726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8b1a5-9089-43ee-b101-f5dfa6a5e1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:44.406353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8b1a5-9089-43ee-b101-f5dfa6a5e1fe None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:44.406711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8b1a5-9089-43ee-b101-f5dfa6a5e1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:44.406988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8b1a5-9089-43ee-b101-f5dfa6a5e1fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:44.420655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:44.502360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8b1a5-9089-43ee-b101-f5dfa6a5e1fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:44.504034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8b1a5-9089-43ee-b101-f5dfa6a5e1fe None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:44.515941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:44.527968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774b048-c2b0-41e1-b659-462d7bf7634d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:44.578478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a1eea-b35b-4314-88e7-e4d744b8ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:44.578478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a1eea-b35b-4314-88e7-e4d744b8ce5c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:44.578478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a1eea-b35b-4314-88e7-e4d744b8ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:44.578478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a1eea-b35b-4314-88e7-e4d744b8ce5c None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:44.578897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a1eea-b35b-4314-88e7-e4d744b8ce5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:44.578897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a1eea-b35b-4314-88e7-e4d744b8ce5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:44.579347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a1eea-b35b-4314-88e7-e4d744b8ce5c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:44.596888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 320/634] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:44.598197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a40bb-e1c8-40dd-970a-03bc7da89bb8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:44.598517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a40bb-e1c8-40dd-970a-03bc7da89bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:44.598785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a40bb-e1c8-40dd-970a-03bc7da89bb8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:44.600040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 321/635] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:44.600755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a40bb-e1c8-40dd-970a-03bc7da89bb8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:44.645962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 315/636] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:44.646792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8b1a5-9089-43ee-b101-f5dfa6a5e1fe None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:44.723128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:44.741782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a40bb-e1c8-40dd-970a-03bc7da89bb8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:44.784965 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99028e7-39a5-4e52-8793-c5f0cfdbad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:44.785604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99028e7-39a5-4e52-8793-c5f0cfdbad02 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:44.785877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99028e7-39a5-4e52-8793-c5f0cfdbad02 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:44.786149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99028e7-39a5-4e52-8793-c5f0cfdbad02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:44.795763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8b1a5-9089-43ee-b101-f5dfa6a5e1fe None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:44.797597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99028e7-39a5-4e52-8793-c5f0cfdbad02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a99028e7-39a5-4e52-8793-c5f0cfdbad02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:44.803079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:44.805633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 322/637] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:44.806517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99028e7-39a5-4e52-8793-c5f0cfdbad02 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:44.811660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:44.839933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e00259-c8fa-41fe-837c-14fa3cb3a395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:44.840553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e00259-c8fa-41fe-837c-14fa3cb3a395 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:44.840886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e00259-c8fa-41fe-837c-14fa3cb3a395 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:44.841160 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e00259-c8fa-41fe-837c-14fa3cb3a395 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:44.841892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e00259-c8fa-41fe-837c-14fa3cb3a395 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:44.842226 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e00259-c8fa-41fe-837c-14fa3cb3a395 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:44.843692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e00259-c8fa-41fe-837c-14fa3cb3a395 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:44.859701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 316/638] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:44.860458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df41b36-2db3-4417-b8f3-616f62a165b6 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:44.860740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df41b36-2db3-4417-b8f3-616f62a165b6 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:44.861043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df41b36-2db3-4417-b8f3-616f62a165b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:44.884106 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-2df41b36-2db3-4417-b8f3-616f62a165b6 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:44.885316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2df41b36-2db3-4417-b8f3-616f62a165b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:44.957269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99028e7-39a5-4e52-8793-c5f0cfdbad02 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:44.971503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 317/639] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:44.973063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df41b36-2db3-4417-b8f3-616f62a165b6 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:45.012345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f646ba-bc59-41cd-a43d-03007eb10b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:45.014151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f646ba-bc59-41cd-a43d-03007eb10b7b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:45.014735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f646ba-bc59-41cd-a43d-03007eb10b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:45.015266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f646ba-bc59-41cd-a43d-03007eb10b7b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:45.025104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f646ba-bc59-41cd-a43d-03007eb10b7b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:45.026600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f646ba-bc59-41cd-a43d-03007eb10b7b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-23f646ba-bc59-41cd-a43d-03007eb10b7b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:45.040671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:45.041734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:45.042715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:45.042715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:45.042715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:45.042715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:45.042715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:45.042715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:45.042715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:45.042715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:45.042715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 23:30:45.042715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:45.047139 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 323/640] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:45.048173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f646ba-bc59-41cd-a43d-03007eb10b7b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:45.053366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:45.150483 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:45.156888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df41b36-2db3-4417-b8f3-616f62a165b6 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:45.214586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f646ba-bc59-41cd-a43d-03007eb10b7b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:45.228339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de3285-0ac0-4f3e-b7e0-e4accd7cd1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:45.229337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de3285-0ac0-4f3e-b7e0-e4accd7cd1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:45.229819 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de3285-0ac0-4f3e-b7e0-e4accd7cd1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:45.230266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de3285-0ac0-4f3e-b7e0-e4accd7cd1a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:45.247947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de3285-0ac0-4f3e-b7e0-e4accd7cd1a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:45.250661 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de3285-0ac0-4f3e-b7e0-e4accd7cd1a9 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:45.268917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379f97f-245f-44dc-a5d2-f7e837281fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:45.269542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f97f-245f-44dc-a5d2-f7e837281fde None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:45.269815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f97f-245f-44dc-a5d2-f7e837281fde None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:45.270086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f97f-245f-44dc-a5d2-f7e837281fde None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:45.270799 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379f97f-245f-44dc-a5d2-f7e837281fde None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:45.271085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379f97f-245f-44dc-a5d2-f7e837281fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:45.272481 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379f97f-245f-44dc-a5d2-f7e837281fde None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-91de3285-0ac0-4f3e-b7e0-e4accd7cd1a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:45.278460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:45.279556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:45.280463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:45.280463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:45.280463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:45.280463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 318/641] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:45.281155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de3285-0ac0-4f3e-b7e0-e4accd7cd1a9 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:45.284741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 324/642] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:45.285763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379f97f-245f-44dc-a5d2-f7e837281fde None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:45.390462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:45.414327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de3285-0ac0-4f3e-b7e0-e4accd7cd1a9 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:45.443845 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379f97f-245f-44dc-a5d2-f7e837281fde None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:45.460272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0baaa46-bcae-4dd3-9a9e-2e5c7bb9f467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:45.461275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baaa46-bcae-4dd3-9a9e-2e5c7bb9f467 None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:45.461607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baaa46-bcae-4dd3-9a9e-2e5c7bb9f467 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:45.461880 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baaa46-bcae-4dd3-9a9e-2e5c7bb9f467 None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/c6b37b09a1ae45e8bc16adfa59500eb9/roles/f0926be4d3d4462fac854212e101ccdd` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:45.482198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be58b0-e8dd-41df-9061-d1ccc841e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:45.482852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be58b0-e8dd-41df-9061-d1ccc841e8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:45.483188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be58b0-e8dd-41df-9061-d1ccc841e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:45.483529 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be58b0-e8dd-41df-9061-d1ccc841e8f6 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:45.484201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0baaa46-bcae-4dd3-9a9e-2e5c7bb9f467 None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=c6b37b09a1ae45e8bc16adfa59500eb9, role_id=f0926be4d3d4462fac854212e101ccdd)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:45.484377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be58b0-e8dd-41df-9061-d1ccc841e8f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:45.484616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be58b0-e8dd-41df-9061-d1ccc841e8f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:45.485668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0baaa46-bcae-4dd3-9a9e-2e5c7bb9f467 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:45.486007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be58b0-e8dd-41df-9061-d1ccc841e8f6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:45.506111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 325/643] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Aug 29 23:30:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:45.507073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be58b0-e8dd-41df-9061-d1ccc841e8f6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:45.579691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:45.584160 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 319/644] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:30:45 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/c6b37b09a1ae45e8bc16adfa59500eb9/roles/f0926be4d3d4462fac854212e101ccdd => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:30:45.588417 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0baaa46-bcae-4dd3-9a9e-2e5c7bb9f467 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:45.650036 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be58b0-e8dd-41df-9061-d1ccc841e8f6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:45.704659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:45.725202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba8ef0-5e95-49c2-a97e-bac5521d5020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:45.726626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba8ef0-5e95-49c2-a97e-bac5521d5020 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:45.727825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba8ef0-5e95-49c2-a97e-bac5521d5020 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:45.728013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba8ef0-5e95-49c2-a97e-bac5521d5020 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:45.732955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0baaa46-bcae-4dd3-9a9e-2e5c7bb9f467 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:45.759191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba8ef0-5e95-49c2-a97e-bac5521d5020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-f0ba8ef0-5e95-49c2-a97e-bac5521d5020 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:45.771645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:45.773333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 326/645] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:45.776223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba8ef0-5e95-49c2-a97e-bac5521d5020 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:45.812446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e608a17-1641-4a35-9e94-3f63ba2b5af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:45.813563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e608a17-1641-4a35-9e94-3f63ba2b5af3 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:45.814167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e608a17-1641-4a35-9e94-3f63ba2b5af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:45.815515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e608a17-1641-4a35-9e94-3f63ba2b5af3 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:45.817855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e608a17-1641-4a35-9e94-3f63ba2b5af3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:45.817855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e608a17-1641-4a35-9e94-3f63ba2b5af3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:45.819610 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e608a17-1641-4a35-9e94-3f63ba2b5af3 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:45.842350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 320/646] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:45.843764 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e608a17-1641-4a35-9e94-3f63ba2b5af3 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:45.930586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba8ef0-5e95-49c2-a97e-bac5521d5020 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:45.952133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:45.976121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08015d8-693a-4ede-a01e-a5f9a661c62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:45.977186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08015d8-693a-4ede-a01e-a5f9a661c62e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:45.977627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08015d8-693a-4ede-a01e-a5f9a661c62e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:45.977900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08015d8-693a-4ede-a01e-a5f9a661c62e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:45.990763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08015d8-693a-4ede-a01e-a5f9a661c62e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:45.992180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08015d8-693a-4ede-a01e-a5f9a661c62e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d08015d8-693a-4ede-a01e-a5f9a661c62e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:46.014823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:46.015847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:46.016750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:46.016750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:46.016750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:46.017478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 327/647] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:46.018457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08015d8-693a-4ede-a01e-a5f9a661c62e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:46.028886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e608a17-1641-4a35-9e94-3f63ba2b5af3 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:46.097098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2941b09-4de9-4e97-a2a4-349ed96c7d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:46.098042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2941b09-4de9-4e97-a2a4-349ed96c7d95 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:46.098673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2941b09-4de9-4e97-a2a4-349ed96c7d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:46.099119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2941b09-4de9-4e97-a2a4-349ed96c7d95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:46.111904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:46.240684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08015d8-693a-4ede-a01e-a5f9a661c62e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:46.252249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2941b09-4de9-4e97-a2a4-349ed96c7d95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:46.254014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2941b09-4de9-4e97-a2a4-349ed96c7d95 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:46.316650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a245c-1f41-4d0c-b512-7faf3acaebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:46.318253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a245c-1f41-4d0c-b512-7faf3acaebf1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:46.323151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a245c-1f41-4d0c-b512-7faf3acaebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:46.324013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a245c-1f41-4d0c-b512-7faf3acaebf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:46.324959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a245c-1f41-4d0c-b512-7faf3acaebf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:46.325587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a245c-1f41-4d0c-b512-7faf3acaebf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:46.328720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a245c-1f41-4d0c-b512-7faf3acaebf1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:46.343887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:46.351544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 328/648] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:46.352924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a245c-1f41-4d0c-b512-7faf3acaebf1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:46.457263 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 321/649] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:46.458327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2941b09-4de9-4e97-a2a4-349ed96c7d95 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:46.597589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:46.605045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a245c-1f41-4d0c-b512-7faf3acaebf1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:46.673248 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94231254-be3f-45f8-9419-84ceb08681c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:46.674569 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94231254-be3f-45f8-9419-84ceb08681c6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:46.675218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94231254-be3f-45f8-9419-84ceb08681c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:46.675789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94231254-be3f-45f8-9419-84ceb08681c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:46.676609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2941b09-4de9-4e97-a2a4-349ed96c7d95 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:46.691892 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94231254-be3f-45f8-9419-84ceb08681c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:46.696567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94231254-be3f-45f8-9419-84ceb08681c6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-94231254-be3f-45f8-9419-84ceb08681c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:46.723749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:46.725663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:46.727635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:46.727635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:46.727635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:46.727635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:46.727635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:46.727635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:46.727635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:46.727635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:46.732573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 329/650] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:46.735135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94231254-be3f-45f8-9419-84ceb08681c6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:46.758685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233cd27-37af-46fe-a6d3-2e470c3b2760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:46.759535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233cd27-37af-46fe-a6d3-2e470c3b2760 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:46.759964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233cd27-37af-46fe-a6d3-2e470c3b2760 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:46.760697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233cd27-37af-46fe-a6d3-2e470c3b2760 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:46.761065 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233cd27-37af-46fe-a6d3-2e470c3b2760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:46.761469 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233cd27-37af-46fe-a6d3-2e470c3b2760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:46.762979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233cd27-37af-46fe-a6d3-2e470c3b2760 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:46.786809 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 322/651] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:46.788852 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233cd27-37af-46fe-a6d3-2e470c3b2760 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:46.830728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:46.975314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94231254-be3f-45f8-9419-84ceb08681c6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:47.007466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233cd27-37af-46fe-a6d3-2e470c3b2760 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:47.048994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4823c8-4e38-4f1c-97d9-e7d8b780fb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:47.050862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4823c8-4e38-4f1c-97d9-e7d8b780fb90 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:47.051700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4823c8-4e38-4f1c-97d9-e7d8b780fb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:47.051700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4823c8-4e38-4f1c-97d9-e7d8b780fb90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:47.069907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4823c8-4e38-4f1c-97d9-e7d8b780fb90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:47.072964 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4823c8-4e38-4f1c-97d9-e7d8b780fb90 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:47.075127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64f6af-7802-4626-9a45-7058419aea07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:47.076584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64f6af-7802-4626-9a45-7058419aea07 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:47.077254 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64f6af-7802-4626-9a45-7058419aea07 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:47.077960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64f6af-7802-4626-9a45-7058419aea07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-3f4823c8-4e38-4f1c-97d9-e7d8b780fb90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:47.099301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:47.101379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:47.103641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:47.103641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:47.103641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:47.103970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64f6af-7802-4626-9a45-7058419aea07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:47.106479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64f6af-7802-4626-9a45-7058419aea07 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:47.107169 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 330/652] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:47.110200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4823c8-4e38-4f1c-97d9-e7d8b780fb90 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:47.143515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-7d64f6af-7802-4626-9a45-7058419aea07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:47.148136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:47.150104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:47.154505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:47.154505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:47.154505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:47.161840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 323/653] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:47.165571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64f6af-7802-4626-9a45-7058419aea07 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:47.332086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4823c8-4e38-4f1c-97d9-e7d8b780fb90 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:47.358983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:47.419424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf09eec-372c-4557-bdb8-16397c0f98f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:47.420380 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf09eec-372c-4557-bdb8-16397c0f98f0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:47.420852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf09eec-372c-4557-bdb8-16397c0f98f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:47.421318 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf09eec-372c-4557-bdb8-16397c0f98f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:47.423327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf09eec-372c-4557-bdb8-16397c0f98f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:47.423777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64f6af-7802-4626-9a45-7058419aea07 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:47.424131 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf09eec-372c-4557-bdb8-16397c0f98f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:47.426780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf09eec-372c-4557-bdb8-16397c0f98f0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:47.478142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 331/654] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:47.480188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf09eec-372c-4557-bdb8-16397c0f98f0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:47.496653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a4937-e133-41b1-b848-a03909873c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:47.498619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a4937-e133-41b1-b848-a03909873c6e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:47.499307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a4937-e133-41b1-b848-a03909873c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:47.499981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a4937-e133-41b1-b848-a03909873c6e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:47.516444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a4937-e133-41b1-b848-a03909873c6e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:47.520722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a4937-e133-41b1-b848-a03909873c6e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-195a4937-e133-41b1-b848-a03909873c6e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:47.544901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:47.546991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:47.549170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:47.549170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:47.549170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:47.549170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:47.549170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:47.549170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:47.549170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:47.549170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:47.549170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 23:30:47.549170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:47.550608 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 324/655] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:47.552249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a4937-e133-41b1-b848-a03909873c6e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:47.669780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:47.724466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf09eec-372c-4557-bdb8-16397c0f98f0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:47.756987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a4937-e133-41b1-b848-a03909873c6e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:47.784600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02ef23-c5cb-4af2-aefa-9be34c3d2d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:47.785566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02ef23-c5cb-4af2-aefa-9be34c3d2d93 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:47.786040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02ef23-c5cb-4af2-aefa-9be34c3d2d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:47.786681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02ef23-c5cb-4af2-aefa-9be34c3d2d93 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:47.787803 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02ef23-c5cb-4af2-aefa-9be34c3d2d93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:47.788323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02ef23-c5cb-4af2-aefa-9be34c3d2d93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:47.790758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02ef23-c5cb-4af2-aefa-9be34c3d2d93 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:47.811971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 332/656] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:47.813336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02ef23-c5cb-4af2-aefa-9be34c3d2d93 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:47.822168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:47.829293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f39cf-276f-4312-ad99-ba39332e2795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:47.830922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f39cf-276f-4312-ad99-ba39332e2795 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:47.831415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f39cf-276f-4312-ad99-ba39332e2795 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:47.835622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f39cf-276f-4312-ad99-ba39332e2795 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:47.835622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f39cf-276f-4312-ad99-ba39332e2795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:47.835622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f39cf-276f-4312-ad99-ba39332e2795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:47.836115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f39cf-276f-4312-ad99-ba39332e2795 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:47.862616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 325/657] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:47.864038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f39cf-276f-4312-ad99-ba39332e2795 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:47.964033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02ef23-c5cb-4af2-aefa-9be34c3d2d93 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:48.009529 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:48.019529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f39cf-276f-4312-ad99-ba39332e2795 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:48.020094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8110d72-1353-4314-91bb-56377519d1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:48.026113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8110d72-1353-4314-91bb-56377519d1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:48.026113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8110d72-1353-4314-91bb-56377519d1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:48.026113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8110d72-1353-4314-91bb-56377519d1a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:48.026113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8110d72-1353-4314-91bb-56377519d1a9 None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=e4a8261d92064b838fc6e8f415879566) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:48.026113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8110d72-1353-4314-91bb-56377519d1a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:48.026907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8110d72-1353-4314-91bb-56377519d1a9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:48.050471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 333/658] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:30:47 2022] GET /identity/v3/role_assignments?user.id=e4a8261d92064b838fc6e8f415879566&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:48.050471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93475-8ef6-4451-b332-bb9675206d93 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:48.050471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93475-8ef6-4451-b332-bb9675206d93 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:48.050471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93475-8ef6-4451-b332-bb9675206d93 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:48.050471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 334/659] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:48.050471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b93475-8ef6-4451-b332-bb9675206d93 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:48.116904 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d82b2c-291d-45a0-a97d-37c121477a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:48.117655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d82b2c-291d-45a0-a97d-37c121477a93 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:48.118002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d82b2c-291d-45a0-a97d-37c121477a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:48.118303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d82b2c-291d-45a0-a97d-37c121477a93 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:48.119101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d82b2c-291d-45a0-a97d-37c121477a93 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:48.119421 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d82b2c-291d-45a0-a97d-37c121477a93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:48.121150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d82b2c-291d-45a0-a97d-37c121477a93 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:48.148326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 326/660] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Aug 29 23:30:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:48.154381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d82b2c-291d-45a0-a97d-37c121477a93 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:48.228167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:48.246865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b93475-8ef6-4451-b332-bb9675206d93 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:48.301428 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83cbeb7-fdd5-486b-8692-8b1d0099969e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:48.302738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83cbeb7-fdd5-486b-8692-8b1d0099969e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:48.303803 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83cbeb7-fdd5-486b-8692-8b1d0099969e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:48.303803 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83cbeb7-fdd5-486b-8692-8b1d0099969e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:48.322766 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83cbeb7-fdd5-486b-8692-8b1d0099969e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:48.325605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83cbeb7-fdd5-486b-8692-8b1d0099969e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:48.348010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d82b2c-291d-45a0-a97d-37c121477a93 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d83cbeb7-fdd5-486b-8692-8b1d0099969e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:48.351343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:48.352850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 335/661] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aa802-8258-46d3-864e-d5e0ae380cdc None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:48.356630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aa802-8258-46d3-864e-d5e0ae380cdc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:48.358061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aa802-8258-46d3-864e-d5e0ae380cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:48.401907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d40aa802-8258-46d3-864e-d5e0ae380cdc None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:48.403197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d40aa802-8258-46d3-864e-d5e0ae380cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:48.457451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81502136-1af5-43fa-8091-30e93dd930f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:48.460672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81502136-1af5-43fa-8091-30e93dd930f3 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:48.460672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81502136-1af5-43fa-8091-30e93dd930f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:48.460672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81502136-1af5-43fa-8091-30e93dd930f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:48.482850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81502136-1af5-43fa-8091-30e93dd930f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:48.485553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:48.487449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81502136-1af5-43fa-8091-30e93dd930f3 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-81502136-1af5-43fa-8091-30e93dd930f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:48.509277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:48.511626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:48.513486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:48.513486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:48.513486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:48.513486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:48.513486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:48.513486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:48.513486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:48.513486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:48.517075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 327/662] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:48.520488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81502136-1af5-43fa-8091-30e93dd930f3 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:48.530854 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 336/663] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:48.531934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40aa802-8258-46d3-864e-d5e0ae380cdc None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:48.730164 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40aa802-8258-46d3-864e-d5e0ae380cdc None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:48.754112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81502136-1af5-43fa-8091-30e93dd930f3 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:48.771191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:48.791578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd6479-a936-40b0-9106-69688b0194cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:48.799068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd6479-a936-40b0-9106-69688b0194cd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:48.799068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd6479-a936-40b0-9106-69688b0194cd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:48.799068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd6479-a936-40b0-9106-69688b0194cd None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:48.799068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd6479-a936-40b0-9106-69688b0194cd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:48.799068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd6479-a936-40b0-9106-69688b0194cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:48.799068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd6479-a936-40b0-9106-69688b0194cd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:48.825263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 337/664] 10.222.0.58 () {56 vars in 1139 bytes} [Mon Aug 29 23:30:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:48.825263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd6479-a936-40b0-9106-69688b0194cd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:48.840140 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f14f4-cc6a-4047-9f5e-b427193c754a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:48.842400 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f14f4-cc6a-4047-9f5e-b427193c754a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:48.843149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f14f4-cc6a-4047-9f5e-b427193c754a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:48.843833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f14f4-cc6a-4047-9f5e-b427193c754a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:48.880101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:48.950797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd6479-a936-40b0-9106-69688b0194cd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:48.988692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f14f4-cc6a-4047-9f5e-b427193c754a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:48.990276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f14f4-cc6a-4047-9f5e-b427193c754a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:49.036713 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8298cd-e7a4-46aa-ba77-828a4354efbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:49.038126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8298cd-e7a4-46aa-ba77-828a4354efbf None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:49.038849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8298cd-e7a4-46aa-ba77-828a4354efbf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:49.039581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8298cd-e7a4-46aa-ba77-828a4354efbf None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:49.041186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8298cd-e7a4-46aa-ba77-828a4354efbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:49.042782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8298cd-e7a4-46aa-ba77-828a4354efbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:49.047480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8298cd-e7a4-46aa-ba77-828a4354efbf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:49.061945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 338/665] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:49.062846 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8298cd-e7a4-46aa-ba77-828a4354efbf None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:49.129010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 328/666] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:49.130951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f14f4-cc6a-4047-9f5e-b427193c754a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:49.150339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:49.201518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8298cd-e7a4-46aa-ba77-828a4354efbf None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:49.203895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:49.250869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a73f9-ba0c-41b0-b8a0-45ce18769ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:49.251980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a73f9-ba0c-41b0-b8a0-45ce18769ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:49.252487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a73f9-ba0c-41b0-b8a0-45ce18769ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:49.252930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a73f9-ba0c-41b0-b8a0-45ce18769ad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:49.286563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f14f4-cc6a-4047-9f5e-b427193c754a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:49.356752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f0a11-d74e-4a76-bf3d-ed58b064983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:49.357397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0a11-d74e-4a76-bf3d-ed58b064983d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:49.357679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0a11-d74e-4a76-bf3d-ed58b064983d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:49.357944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0a11-d74e-4a76-bf3d-ed58b064983d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:49.370846 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a73f9-ba0c-41b0-b8a0-45ce18769ad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:49.371824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f0a11-d74e-4a76-bf3d-ed58b064983d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:49.373854 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a73f9-ba0c-41b0-b8a0-45ce18769ad3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-6f3f0a11-d74e-4a76-bf3d-ed58b064983d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:49.377903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:49.379662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:49.381595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:49.384024 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 329/667] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:49.385020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f0a11-d74e-4a76-bf3d-ed58b064983d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:49.477563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:49.528039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 339/668] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:49.528927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a73f9-ba0c-41b0-b8a0-45ce18769ad3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:49.530680 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f0a11-d74e-4a76-bf3d-ed58b064983d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:49.539862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:49.605127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c41e15-4f9a-49a7-bd67-31fa62c717fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:49.607826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c41e15-4f9a-49a7-bd67-31fa62c717fa None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:49.608307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c41e15-4f9a-49a7-bd67-31fa62c717fa None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:49.608705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c41e15-4f9a-49a7-bd67-31fa62c717fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:49.609600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c41e15-4f9a-49a7-bd67-31fa62c717fa None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=d7a59337d20e4678b8a16ac35bffbdb4) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:49.610112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c41e15-4f9a-49a7-bd67-31fa62c717fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:49.613234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c41e15-4f9a-49a7-bd67-31fa62c717fa None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:49.634460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 330/669] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:30:49 2022] GET /identity/v3/role_assignments?user.id=d7a59337d20e4678b8a16ac35bffbdb4&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:49.634460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c41e15-4f9a-49a7-bd67-31fa62c717fa None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:49.701518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a73f9-ba0c-41b0-b8a0-45ce18769ad3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:49.755387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1f95e-30b0-424c-a73f-e627273beed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:49.756160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f95e-30b0-424c-a73f-e627273beed4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:49.756336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f95e-30b0-424c-a73f-e627273beed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:49.756619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f95e-30b0-424c-a73f-e627273beed4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:49.778850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1f95e-30b0-424c-a73f-e627273beed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:49.783687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1f95e-30b0-424c-a73f-e627273beed4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-3ea1f95e-30b0-424c-a73f-e627273beed4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:49.803809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:49.804889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:49.810967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:49.810967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:49.810967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:49.810967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 340/670] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:49.810967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1f95e-30b0-424c-a73f-e627273beed4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:49.817238 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c41e15-4f9a-49a7-bd67-31fa62c717fa None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:49.857738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055307b-d2eb-42a9-97b4-50a92cf1957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:49.858763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055307b-d2eb-42a9-97b4-50a92cf1957f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:49.859051 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055307b-d2eb-42a9-97b4-50a92cf1957f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:49.859314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055307b-d2eb-42a9-97b4-50a92cf1957f None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:49.859995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055307b-d2eb-42a9-97b4-50a92cf1957f None admin] RBAC: Adding query filter params (name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:49.860307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055307b-d2eb-42a9-97b4-50a92cf1957f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:49.862818 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055307b-d2eb-42a9-97b4-50a92cf1957f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:49.864982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:49.875649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 331/671] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:49.882526 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055307b-d2eb-42a9-97b4-50a92cf1957f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:49.914153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:50.003012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1f95e-30b0-424c-a73f-e627273beed4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:50.060278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e0911-3901-4fd3-9b5f-701fbfb135a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:50.060278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e0911-3901-4fd3-9b5f-701fbfb135a0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:50.060278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e0911-3901-4fd3-9b5f-701fbfb135a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:50.060278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e0911-3901-4fd3-9b5f-701fbfb135a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:50.088353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e0911-3901-4fd3-9b5f-701fbfb135a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:50.089833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e0911-3901-4fd3-9b5f-701fbfb135a0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:50.110220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055307b-d2eb-42a9-97b4-50a92cf1957f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-498e0911-3901-4fd3-9b5f-701fbfb135a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:50.124620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:50.126520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:50.128294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:50.128294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:50.128294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:50.128294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 341/672] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:50.128294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e0911-3901-4fd3-9b5f-701fbfb135a0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:50.204943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd23b9a-7f57-42f9-8e37-68f61c607529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:50.205413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd23b9a-7f57-42f9-8e37-68f61c607529 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:50.205413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd23b9a-7f57-42f9-8e37-68f61c607529 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:50.205900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd23b9a-7f57-42f9-8e37-68f61c607529 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:50.210455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd23b9a-7f57-42f9-8e37-68f61c607529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:50.210455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd23b9a-7f57-42f9-8e37-68f61c607529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:50.210455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd23b9a-7f57-42f9-8e37-68f61c607529 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:50.237234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 332/673] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:50.239286 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd23b9a-7f57-42f9-8e37-68f61c607529 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:50.299649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e0911-3901-4fd3-9b5f-701fbfb135a0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:50.307731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:50.329558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:50.349208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9127b-5a76-46e9-b9eb-b231f94aeccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:50.350005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9127b-5a76-46e9-b9eb-b231f94aeccf None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:50.350281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9127b-5a76-46e9-b9eb-b231f94aeccf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:50.350658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9127b-5a76-46e9-b9eb-b231f94aeccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:50.363770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9127b-5a76-46e9-b9eb-b231f94aeccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:50.365234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9127b-5a76-46e9-b9eb-b231f94aeccf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e5d9127b-5a76-46e9-b9eb-b231f94aeccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:50.382611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:50.384477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:50.386225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:50.386225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:50.386225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:50.386225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 342/674] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:50.386225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9127b-5a76-46e9-b9eb-b231f94aeccf None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:50.421149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd23b9a-7f57-42f9-8e37-68f61c607529 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:50.481644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d31af-0977-4130-8c00-b9852349bfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:50.483321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d31af-0977-4130-8c00-b9852349bfd3 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:50.484145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d31af-0977-4130-8c00-b9852349bfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:50.484864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d31af-0977-4130-8c00-b9852349bfd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:50.486921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d31af-0977-4130-8c00-b9852349bfd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:50.490128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d31af-0977-4130-8c00-b9852349bfd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:50.490881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d31af-0977-4130-8c00-b9852349bfd3 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:50.502393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9127b-5a76-46e9-b9eb-b231f94aeccf None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:50.511459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 333/675] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:50.513322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d31af-0977-4130-8c00-b9852349bfd3 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:50.541973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31218-5b29-4675-93a5-15e2fc3f2a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:50.542644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31218-5b29-4675-93a5-15e2fc3f2a41 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:50.542920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31218-5b29-4675-93a5-15e2fc3f2a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:50.543188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31218-5b29-4675-93a5-15e2fc3f2a41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:50.551800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef31218-5b29-4675-93a5-15e2fc3f2a41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:50.553279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef31218-5b29-4675-93a5-15e2fc3f2a41 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-0ef31218-5b29-4675-93a5-15e2fc3f2a41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:50.566344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:50.568061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:50.569642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:50.569642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:50.569642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:50.569642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:50.569642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:50.569642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:50.569642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:50.569642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:50.569642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 343/676] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:50.569642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31218-5b29-4675-93a5-15e2fc3f2a41 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:50.590540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:50.671969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d31af-0977-4130-8c00-b9852349bfd3 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:50.689581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31218-5b29-4675-93a5-15e2fc3f2a41 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:50.691692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:50.712275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06acc6-f772-4c64-8f9a-b485a0768fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:50.712911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06acc6-f772-4c64-8f9a-b485a0768fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:50.713190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06acc6-f772-4c64-8f9a-b485a0768fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:50.713480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06acc6-f772-4c64-8f9a-b485a0768fd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:50.714159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06acc6-f772-4c64-8f9a-b485a0768fd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:50.714486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06acc6-f772-4c64-8f9a-b485a0768fd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:50.715957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06acc6-f772-4c64-8f9a-b485a0768fd4 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:50.730131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 334/677] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:50.730883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06acc6-f772-4c64-8f9a-b485a0768fd4 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:50.746265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc25c78-565c-4108-9784-7680a7389e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:50.747197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc25c78-565c-4108-9784-7680a7389e41 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:50.747624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc25c78-565c-4108-9784-7680a7389e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:50.748039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc25c78-565c-4108-9784-7680a7389e41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:50.758545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc25c78-565c-4108-9784-7680a7389e41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:50.760849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc25c78-565c-4108-9784-7680a7389e41 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-3bc25c78-565c-4108-9784-7680a7389e41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:50.775817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:50.776809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:50.777738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:50.777738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:50.777738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:50.777738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:50.777738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:50.777738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:50.777738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:50.777738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:50.779907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 344/678] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:50.781257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53217268-997b-47a8-b4e3-70c7015ec36a None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:50.781549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53217268-997b-47a8-b4e3-70c7015ec36a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:50.781802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53217268-997b-47a8-b4e3-70c7015ec36a None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:50.783710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 345/679] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:50.784275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53217268-997b-47a8-b4e3-70c7015ec36a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:50.901322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06acc6-f772-4c64-8f9a-b485a0768fd4 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:50.905977 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:50.937772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53217268-997b-47a8-b4e3-70c7015ec36a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:50.958467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac69df-292a-4f0c-a3a9-a5b67e344ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:50.958467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac69df-292a-4f0c-a3a9-a5b67e344ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:50.958467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac69df-292a-4f0c-a3a9-a5b67e344ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:50.958467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac69df-292a-4f0c-a3a9-a5b67e344ba8 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:50.958944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac69df-292a-4f0c-a3a9-a5b67e344ba8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:50.959131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac69df-292a-4f0c-a3a9-a5b67e344ba8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:50.960601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac69df-292a-4f0c-a3a9-a5b67e344ba8 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:50.974922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 335/680] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:50.980889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f159b-f397-4aba-8600-d0da97f698e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:50.981395 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f159b-f397-4aba-8600-d0da97f698e6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:50.981674 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f159b-f397-4aba-8600-d0da97f698e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:50.981946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f159b-f397-4aba-8600-d0da97f698e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:50.982442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac69df-292a-4f0c-a3a9-a5b67e344ba8 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:50.995825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f159b-f397-4aba-8600-d0da97f698e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:50.998244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f159b-f397-4aba-8600-d0da97f698e6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-b90f159b-f397-4aba-8600-d0da97f698e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:51.021138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:51.022591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:51.023746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:51.023746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:51.023746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:51.023746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:51.023746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:51.023746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:51.023746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:51.023746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:51.023746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:30:51.023746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:51.026791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 346/681] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:30:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:51.028629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1531b-4239-438f-9463-802707e1756b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:51.029223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1531b-4239-438f-9463-802707e1756b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:51.029691 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1531b-4239-438f-9463-802707e1756b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:51.046504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:51.050379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-fbf1531b-4239-438f-9463-802707e1756b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:51.051643 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fbf1531b-4239-438f-9463-802707e1756b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:51.133178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac69df-292a-4f0c-a3a9-a5b67e344ba8 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:51.142787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 347/682] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:51.142787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1531b-4239-438f-9463-802707e1756b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:51.195918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf202b8-beec-40a6-8f24-c4acda040d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:51.197231 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf202b8-beec-40a6-8f24-c4acda040d12 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:51.197871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf202b8-beec-40a6-8f24-c4acda040d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:51.198557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf202b8-beec-40a6-8f24-c4acda040d12 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:51.199946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf202b8-beec-40a6-8f24-c4acda040d12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:51.200755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf202b8-beec-40a6-8f24-c4acda040d12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:51.203492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf202b8-beec-40a6-8f24-c4acda040d12 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:51.225725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 336/683] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:51.227404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf202b8-beec-40a6-8f24-c4acda040d12 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:51.265461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:51.336324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1531b-4239-438f-9463-802707e1756b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:51.404340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:51.410814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2a208-17f2-4cba-851f-23b4d0206e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:51.412255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2a208-17f2-4cba-851f-23b4d0206e0d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:51.412955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2a208-17f2-4cba-851f-23b4d0206e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:51.413623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2a208-17f2-4cba-851f-23b4d0206e0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:51.414932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf202b8-beec-40a6-8f24-c4acda040d12 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:51.415508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2a208-17f2-4cba-851f-23b4d0206e0d None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=64a42591a6814a0cbcb8444829cb25b3) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:51.416215 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2a208-17f2-4cba-851f-23b4d0206e0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:51.418981 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2a208-17f2-4cba-851f-23b4d0206e0d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:51.427986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 348/684] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:30:51 2022] GET /identity/v3/role_assignments?user.id=64a42591a6814a0cbcb8444829cb25b3&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:51.429287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2a208-17f2-4cba-851f-23b4d0206e0d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:51.492956 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d0ed9-956e-41af-8d9d-7a0b5778a704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:51.494059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d0ed9-956e-41af-8d9d-7a0b5778a704 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:51.494811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d0ed9-956e-41af-8d9d-7a0b5778a704 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:51.495473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d0ed9-956e-41af-8d9d-7a0b5778a704 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:51.497471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d0ed9-956e-41af-8d9d-7a0b5778a704 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:51.498032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d0ed9-956e-41af-8d9d-7a0b5778a704 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:51.501777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d0ed9-956e-41af-8d9d-7a0b5778a704 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:51.549120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 337/685] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Aug 29 23:30:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:51.551624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d0ed9-956e-41af-8d9d-7a0b5778a704 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:51.592268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2a208-17f2-4cba-851f-23b4d0206e0d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:51.644682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:51.652630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3f571-c931-42c9-bfcb-69874e6bdd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:51.654263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3f571-c931-42c9-bfcb-69874e6bdd40 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:51.654730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3f571-c931-42c9-bfcb-69874e6bdd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:51.663149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3f571-c931-42c9-bfcb-69874e6bdd40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:51.663149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3f571-c931-42c9-bfcb-69874e6bdd40 None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=863bcfa6b8694182b9d163133a141907) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:51.663149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3f571-c931-42c9-bfcb-69874e6bdd40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:51.663149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3f571-c931-42c9-bfcb-69874e6bdd40 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:51.672058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 349/686] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:30:51 2022] GET /identity/v3/role_assignments?user.id=863bcfa6b8694182b9d163133a141907&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:51.680692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3f571-c931-42c9-bfcb-69874e6bdd40 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:51.693245 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d0ed9-956e-41af-8d9d-7a0b5778a704 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:51.742247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20376963-3ee1-4fbf-9d12-eac15c13509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:51.742948 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20376963-3ee1-4fbf-9d12-eac15c13509b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:51.743517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20376963-3ee1-4fbf-9d12-eac15c13509b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:51.743842 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20376963-3ee1-4fbf-9d12-eac15c13509b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:51.763648 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20376963-3ee1-4fbf-9d12-eac15c13509b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-20376963-3ee1-4fbf-9d12-eac15c13509b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:51.773487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:51.775399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:51.777017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 338/687] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:30:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:51.787281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20376963-3ee1-4fbf-9d12-eac15c13509b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:51.817496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:51.944342 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20376963-3ee1-4fbf-9d12-eac15c13509b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:51.956734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3f571-c931-42c9-bfcb-69874e6bdd40 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:51.994185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99497be7-f2d1-4293-be6d-e22cf8a0f27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:51.994770 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99497be7-f2d1-4293-be6d-e22cf8a0f27d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:51.995041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99497be7-f2d1-4293-be6d-e22cf8a0f27d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:51.995315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99497be7-f2d1-4293-be6d-e22cf8a0f27d None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:51.996020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99497be7-f2d1-4293-be6d-e22cf8a0f27d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:51.996393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99497be7-f2d1-4293-be6d-e22cf8a0f27d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:51.997805 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99497be7-f2d1-4293-be6d-e22cf8a0f27d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:52.013243 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 339/688] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:52.014921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99497be7-f2d1-4293-be6d-e22cf8a0f27d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:52.051941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0058d-c51b-4e01-a50c-b5819bdcbc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:52.052652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0058d-c51b-4e01-a50c-b5819bdcbc41 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:52.052934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0058d-c51b-4e01-a50c-b5819bdcbc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:52.053204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0058d-c51b-4e01-a50c-b5819bdcbc41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:52.156931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:52.167477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0058d-c51b-4e01-a50c-b5819bdcbc41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:52.169004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0058d-c51b-4e01-a50c-b5819bdcbc41 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:52.187046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:52.244049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99497be7-f2d1-4293-be6d-e22cf8a0f27d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:52.331987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 350/689] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:52.341104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0058d-c51b-4e01-a50c-b5819bdcbc41 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:52.345803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b4072-8466-48cf-a76c-8263babcc799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:52.346979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b4072-8466-48cf-a76c-8263babcc799 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:52.347473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b4072-8466-48cf-a76c-8263babcc799 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:52.347961 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b4072-8466-48cf-a76c-8263babcc799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:52.401627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b4072-8466-48cf-a76c-8263babcc799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:52.404094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b4072-8466-48cf-a76c-8263babcc799 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-522b4072-8466-48cf-a76c-8263babcc799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:52.447294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:52.448477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:52.449420 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:52.449420 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:52.449420 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:52.449752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 340/690] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:52.457269 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b4072-8466-48cf-a76c-8263babcc799 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:52.604237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:52.613970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0058d-c51b-4e01-a50c-b5819bdcbc41 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:52.653436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91927a-e281-47b1-a419-4e5400699d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:52.654147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91927a-e281-47b1-a419-4e5400699d48 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:52.654446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91927a-e281-47b1-a419-4e5400699d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:52.654746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91927a-e281-47b1-a419-4e5400699d48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:52.667113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91927a-e281-47b1-a419-4e5400699d48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:52.668673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91927a-e281-47b1-a419-4e5400699d48 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ff91927a-e281-47b1-a419-4e5400699d48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:52.686332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:52.687458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:52.688361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:52.688361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:52.688361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:52.688361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 351/691] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:52.694289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91927a-e281-47b1-a419-4e5400699d48 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:52.795907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b4072-8466-48cf-a76c-8263babcc799 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:52.843387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91927a-e281-47b1-a419-4e5400699d48 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:52.856787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:52.875084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88336ac-1b78-433f-8dc3-b7f453747adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:52.876676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88336ac-1b78-433f-8dc3-b7f453747adb None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:52.877162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88336ac-1b78-433f-8dc3-b7f453747adb None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:52.877797 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88336ac-1b78-433f-8dc3-b7f453747adb None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:52.883000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88336ac-1b78-433f-8dc3-b7f453747adb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:52.883320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88336ac-1b78-433f-8dc3-b7f453747adb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:52.885315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88336ac-1b78-433f-8dc3-b7f453747adb None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:52.896110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7ebfe-db34-41cb-a938-8ae78b47763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:52.896732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7ebfe-db34-41cb-a938-8ae78b47763f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:52.897013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7ebfe-db34-41cb-a938-8ae78b47763f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:52.897284 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7ebfe-db34-41cb-a938-8ae78b47763f None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:52.897948 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7ebfe-db34-41cb-a938-8ae78b47763f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:52.898254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7ebfe-db34-41cb-a938-8ae78b47763f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:52.899692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7ebfe-db34-41cb-a938-8ae78b47763f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:52.916499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 341/692] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:52.919841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 352/693] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:52.923982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c3e09-2462-44d2-808e-37a77c0cec0f None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:52.924397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c3e09-2462-44d2-808e-37a77c0cec0f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:52.924867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c3e09-2462-44d2-808e-37a77c0cec0f None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:52.925766 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7ebfe-db34-41cb-a938-8ae78b47763f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:52.926592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 342/694] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:52.927365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c3e09-2462-44d2-808e-37a77c0cec0f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:52.942795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:53.058207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7ebfe-db34-41cb-a938-8ae78b47763f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:53.076029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c3e09-2462-44d2-808e-37a77c0cec0f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:53.106377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f7109-6027-408b-9bf4-1f62a931578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:53.106988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7109-6027-408b-9bf4-1f62a931578d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:53.107496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7109-6027-408b-9bf4-1f62a931578d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:53.108018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7109-6027-408b-9bf4-1f62a931578d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:53.117889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f7109-6027-408b-9bf4-1f62a931578d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:53.120003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f7109-6027-408b-9bf4-1f62a931578d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:53.127400 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497d72c-6278-49c9-aa3d-466bde925009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:53.128661 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497d72c-6278-49c9-aa3d-466bde925009 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:53.129192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497d72c-6278-49c9-aa3d-466bde925009 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:53.129609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497d72c-6278-49c9-aa3d-466bde925009 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-f08f7109-6027-408b-9bf4-1f62a931578d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:53.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:53.135923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:53.136859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:53.136859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:53.136859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:53.136859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:53.136859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:53.136859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:53.136859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:53.136859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:53.136859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:30:53.136859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:53.136859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 353/695] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Aug 29 23:30:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:53.138909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80325657-1701-41d7-b762-e8795d2af543 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:53.139261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80325657-1701-41d7-b762-e8795d2af543 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:53.139621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80325657-1701-41d7-b762-e8795d2af543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:53.142712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497d72c-6278-49c9-aa3d-466bde925009 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:53.144989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497d72c-6278-49c9-aa3d-466bde925009 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-3497d72c-6278-49c9-aa3d-466bde925009 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:53.162047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:53.163081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:53.163979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:53.163979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:53.163979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:53.163979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:53.163979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:53.163979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:53.163979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:53.163979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:53.166042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-80325657-1701-41d7-b762-e8795d2af543 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:53.166221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 343/696] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:53.167338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497d72c-6278-49c9-aa3d-466bde925009 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:53.172377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-80325657-1701-41d7-b762-e8795d2af543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:53.241146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:53.256463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:53.270460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 354/697] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:53.270460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80325657-1701-41d7-b762-e8795d2af543 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:53.352602 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497d72c-6278-49c9-aa3d-466bde925009 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:53.431060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f821f54-a194-4f0b-bd03-b7210b74b5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:53.431609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f821f54-a194-4f0b-bd03-b7210b74b5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:53.431889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f821f54-a194-4f0b-bd03-b7210b74b5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:53.432162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f821f54-a194-4f0b-bd03-b7210b74b5b6 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:53.433023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f821f54-a194-4f0b-bd03-b7210b74b5b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:53.433332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f821f54-a194-4f0b-bd03-b7210b74b5b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:53.436712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f821f54-a194-4f0b-bd03-b7210b74b5b6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:53.472189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 344/698] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Aug 29 23:30:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:53.479897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f821f54-a194-4f0b-bd03-b7210b74b5b6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:53.480730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80325657-1701-41d7-b762-e8795d2af543 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:53.569585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395bbe4-e662-4797-af75-a137511a66ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:53.570586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395bbe4-e662-4797-af75-a137511a66ae None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:53.571029 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395bbe4-e662-4797-af75-a137511a66ae None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:53.571462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395bbe4-e662-4797-af75-a137511a66ae None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:53.572851 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395bbe4-e662-4797-af75-a137511a66ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:53.573339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395bbe4-e662-4797-af75-a137511a66ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:53.575901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395bbe4-e662-4797-af75-a137511a66ae None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:53.594286 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 355/699] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:53.600729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395bbe4-e662-4797-af75-a137511a66ae None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:53.680549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:53.705297 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f821f54-a194-4f0b-bd03-b7210b74b5b6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:53.716443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:53.746008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395bbe4-e662-4797-af75-a137511a66ae None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:53.756846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c0032-4423-4738-b13f-690f4ab545ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:53.757853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c0032-4423-4738-b13f-690f4ab545ad None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:53.758344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c0032-4423-4738-b13f-690f4ab545ad None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:53.759338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c0032-4423-4738-b13f-690f4ab545ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:53.790189 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55b67a-9412-4f69-a0cc-666a62921e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:53.790901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55b67a-9412-4f69-a0cc-666a62921e5f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:53.791183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55b67a-9412-4f69-a0cc-666a62921e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:53.792001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55b67a-9412-4f69-a0cc-666a62921e5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:53.792760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55b67a-9412-4f69-a0cc-666a62921e5f None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=c6b37b09a1ae45e8bc16adfa59500eb9) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:53.793087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55b67a-9412-4f69-a0cc-666a62921e5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:53.795069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55b67a-9412-4f69-a0cc-666a62921e5f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:53.807518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 356/700] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:30:53 2022] GET /identity/v3/role_assignments?user.id=c6b37b09a1ae45e8bc16adfa59500eb9&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:53.807518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55b67a-9412-4f69-a0cc-666a62921e5f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:53.911128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c0032-4423-4738-b13f-690f4ab545ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:53.913028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c0032-4423-4738-b13f-690f4ab545ad None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:53.950555 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55b67a-9412-4f69-a0cc-666a62921e5f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:53.981377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:53.997525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddfaf2-943b-4cd9-9521-e1a3144a9240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:53.998278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddfaf2-943b-4cd9-9521-e1a3144a9240 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:53.998579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddfaf2-943b-4cd9-9521-e1a3144a9240 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:53.998846 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddfaf2-943b-4cd9-9521-e1a3144a9240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:54.011611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddfaf2-943b-4cd9-9521-e1a3144a9240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:54.012291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddfaf2-943b-4cd9-9521-e1a3144a9240 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d2ddfaf2-943b-4cd9-9521-e1a3144a9240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:54.031508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:54.033226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:54.034134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:54.034134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:54.034134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:54.036460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 357/701] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:54.038054 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba521f-7291-4015-867a-b76a2702b9e6 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:54.038543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba521f-7291-4015-867a-b76a2702b9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:54.038981 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba521f-7291-4015-867a-b76a2702b9e6 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:54.040663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 358/702] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:54.041818 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba521f-7291-4015-867a-b76a2702b9e6 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:54.079438 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 345/703] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:30:54.081443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cc97f-2494-417c-9ea7-2f6669fcfd28 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:54.081996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cc97f-2494-417c-9ea7-2f6669fcfd28 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:54.082289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cc97f-2494-417c-9ea7-2f6669fcfd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:54.105843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:54.118796 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-269cc97f-2494-417c-9ea7-2f6669fcfd28 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:54.121122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-269cc97f-2494-417c-9ea7-2f6669fcfd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:54.178073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba521f-7291-4015-867a-b76a2702b9e6 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:54.213192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 346/704] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:54.214343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cc97f-2494-417c-9ea7-2f6669fcfd28 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:54.234610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0cba2-5e25-4c00-af3b-353c6b4cb999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:54.237712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0cba2-5e25-4c00-af3b-353c6b4cb999 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:54.238171 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0cba2-5e25-4c00-af3b-353c6b4cb999 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:54.238623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0cba2-5e25-4c00-af3b-353c6b4cb999 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:54.239690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0cba2-5e25-4c00-af3b-353c6b4cb999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:54.241000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0cba2-5e25-4c00-af3b-353c6b4cb999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:54.244288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0cba2-5e25-4c00-af3b-353c6b4cb999 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:54.271634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 359/705] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:54.275992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0cba2-5e25-4c00-af3b-353c6b4cb999 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:54.346922 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:54.450628 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cc97f-2494-417c-9ea7-2f6669fcfd28 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:54.459384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0cba2-5e25-4c00-af3b-353c6b4cb999 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:54.525563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa6c33-fdea-4e22-be85-3aa5d176e168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:54.526128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa6c33-fdea-4e22-be85-3aa5d176e168 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:54.526410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa6c33-fdea-4e22-be85-3aa5d176e168 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:54.527803 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f3fa4-714c-4b36-99de-67e8c881a38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:54.528405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f3fa4-714c-4b36-99de-67e8c881a38e None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:54.528683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f3fa4-714c-4b36-99de-67e8c881a38e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:54.528954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f3fa4-714c-4b36-99de-67e8c881a38e None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:54.529609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f3fa4-714c-4b36-99de-67e8c881a38e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:54.531098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f3fa4-714c-4b36-99de-67e8c881a38e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:54.534546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa6c33-fdea-4e22-be85-3aa5d176e168 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:54.537899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:54.552405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa6c33-fdea-4e22-be85-3aa5d176e168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:54.553883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa6c33-fdea-4e22-be85-3aa5d176e168 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-faaa6c33-fdea-4e22-be85-3aa5d176e168 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:54.574156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:54.575356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:54.576357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:54.576357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:54.576357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:54.576357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 347/706] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:54.576624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa6c33-fdea-4e22-be85-3aa5d176e168 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:54.667120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 360/707] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:30:54 2022] POST /identity/v3/users => generated 254 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:30:54.668955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f3fa4-714c-4b36-99de-67e8c881a38e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:54.738039 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa6c33-fdea-4e22-be85-3aa5d176e168 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:54.784191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42576c6-3ab6-4717-9367-bf973dee9543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:54.785065 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42576c6-3ab6-4717-9367-bf973dee9543 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:54.785505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42576c6-3ab6-4717-9367-bf973dee9543 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:54.785912 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42576c6-3ab6-4717-9367-bf973dee9543 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:54.786726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42576c6-3ab6-4717-9367-bf973dee9543 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:54.788963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42576c6-3ab6-4717-9367-bf973dee9543 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-a42576c6-3ab6-4717-9367-bf973dee9543 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:54.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:54.804932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 23:30:54.805957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:54.808734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 348/708] 10.222.0.58 () {56 vars in 1129 bytes} [Mon Aug 29 23:30:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:54.818269 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42576c6-3ab6-4717-9367-bf973dee9543 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:54.897337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:54.914664 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:54.927598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f3fa4-714c-4b36-99de-67e8c881a38e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:54.963912 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42576c6-3ab6-4717-9367-bf973dee9543 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:54.972324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddb3ee-5175-41c0-af33-081e18b8d764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:54.973094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddb3ee-5175-41c0-af33-081e18b8d764 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:54.973473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddb3ee-5175-41c0-af33-081e18b8d764 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:54.973847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddb3ee-5175-41c0-af33-081e18b8d764 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:54.974650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddb3ee-5175-41c0-af33-081e18b8d764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:54.975549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddb3ee-5175-41c0-af33-081e18b8d764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:54.976741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddb3ee-5175-41c0-af33-081e18b8d764 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:54.993042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 361/709] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:54.998811 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddb3ee-5175-41c0-af33-081e18b8d764 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:55.014014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1111448-1140-43c8-9569-83b40c0bafef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:55.014846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1111448-1140-43c8-9569-83b40c0bafef None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:55.015545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1111448-1140-43c8-9569-83b40c0bafef None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:55.015665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1111448-1140-43c8-9569-83b40c0bafef None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:55.016989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1111448-1140-43c8-9569-83b40c0bafef None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:55.017752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1111448-1140-43c8-9569-83b40c0bafef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:55.019495 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1111448-1140-43c8-9569-83b40c0bafef None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:55.030907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 349/710] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Aug 29 23:30:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:55.031794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1111448-1140-43c8-9569-83b40c0bafef None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:55.138615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddb3ee-5175-41c0-af33-081e18b8d764 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:55.170631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1111448-1140-43c8-9569-83b40c0bafef None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:55.184280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a4af7-7b63-4068-a0c0-dc44cddb1bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:55.184925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4af7-7b63-4068-a0c0-dc44cddb1bbd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:55.185206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4af7-7b63-4068-a0c0-dc44cddb1bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:55.185844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4af7-7b63-4068-a0c0-dc44cddb1bbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:55.196739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a4af7-7b63-4068-a0c0-dc44cddb1bbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:55.199584 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a4af7-7b63-4068-a0c0-dc44cddb1bbd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:55.216187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b31e78-0f44-47d5-a1b2-41372b80a10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:55.217080 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31e78-0f44-47d5-a1b2-41372b80a10b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:55.217641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31e78-0f44-47d5-a1b2-41372b80a10b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:55.218077 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31e78-0f44-47d5-a1b2-41372b80a10b None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:55.218982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b31e78-0f44-47d5-a1b2-41372b80a10b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:55.219430 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b31e78-0f44-47d5-a1b2-41372b80a10b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:55.221883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b31e78-0f44-47d5-a1b2-41372b80a10b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-011a4af7-7b63-4068-a0c0-dc44cddb1bbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:30:55.223213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:55.224390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:55.225633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:30:55.225633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:30:55.225633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:30:55.225633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:30:55.225633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:30:55.225633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:30:55.225633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:30:55.225633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:55.225633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 362/711] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:55.226621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:55.228113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464330e-23b8-4d20-af97-f0f6463cae0d None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:55.228280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464330e-23b8-4d20-af97-f0f6463cae0d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:55.228728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464330e-23b8-4d20-af97-f0f6463cae0d None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:55.231983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 363/712] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:55.232299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 350/713] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Aug 29 23:30:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:55.233403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464330e-23b8-4d20-af97-f0f6463cae0d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:55.243340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed26e1-05e0-411e-83ad-4a3265e45df8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:55.243808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed26e1-05e0-411e-83ad-4a3265e45df8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:55.244347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed26e1-05e0-411e-83ad-4a3265e45df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:55.252122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:55.282566 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-34ed26e1-05e0-411e-83ad-4a3265e45df8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:55.283908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-34ed26e1-05e0-411e-83ad-4a3265e45df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:55.450250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 351/714] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:55.502104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464330e-23b8-4d20-af97-f0f6463cae0d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:55.546208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad12e64-ca41-4349-aed6-57a96d166724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:55.546880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad12e64-ca41-4349-aed6-57a96d166724 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:55.547159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad12e64-ca41-4349-aed6-57a96d166724 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:55.547432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad12e64-ca41-4349-aed6-57a96d166724 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:55.548956 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad12e64-ca41-4349-aed6-57a96d166724 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:55.549269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad12e64-ca41-4349-aed6-57a96d166724 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:55.550930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad12e64-ca41-4349-aed6-57a96d166724 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:55.582242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 364/715] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:55.590296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed26e1-05e0-411e-83ad-4a3265e45df8 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:55.794409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed26e1-05e0-411e-83ad-4a3265e45df8 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:55.816465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:55.863041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737725f-783e-4a76-b28e-f35e29556e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:55.864101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737725f-783e-4a76-b28e-f35e29556e9b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:55.864604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737725f-783e-4a76-b28e-f35e29556e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:55.865104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737725f-783e-4a76-b28e-f35e29556e9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:55.866578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737725f-783e-4a76-b28e-f35e29556e9b None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=e4a8261d92064b838fc6e8f415879566) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:55.867151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737725f-783e-4a76-b28e-f35e29556e9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:55.869994 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737725f-783e-4a76-b28e-f35e29556e9b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:55.882452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 352/716] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:30:55 2022] GET /identity/v3/role_assignments?user.id=e4a8261d92064b838fc6e8f415879566&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=d86dba8efe20423abdde06031690beb8 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:56.071970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad12e64-ca41-4349-aed6-57a96d166724 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:56.205457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:56.241046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad12e64-ca41-4349-aed6-57a96d166724 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:56.289829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420865f-f870-4fb7-a4ef-bd8a768498a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:56.290491 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420865f-f870-4fb7-a4ef-bd8a768498a5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:56.290769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420865f-f870-4fb7-a4ef-bd8a768498a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:56.291040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420865f-f870-4fb7-a4ef-bd8a768498a5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:56.291778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420865f-f870-4fb7-a4ef-bd8a768498a5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:56.293165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420865f-f870-4fb7-a4ef-bd8a768498a5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-6420865f-f870-4fb7-a4ef-bd8a768498a5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:56.321830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:56.323824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 23:30:56.325441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:56.334950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 365/717] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:30:56 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:56.344844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737725f-783e-4a76-b28e-f35e29556e9b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:56.381104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2e5e1-fdca-4fa1-bb69-1bf95634bd3a None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:56.381558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2e5e1-fdca-4fa1-bb69-1bf95634bd3a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:56.382009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2e5e1-fdca-4fa1-bb69-1bf95634bd3a None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:56.384919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 366/718] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:56.390695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97f188-8c08-47d8-9033-c626c86a9f37 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:56.391770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97f188-8c08-47d8-9033-c626c86a9f37 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:56.392692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97f188-8c08-47d8-9033-c626c86a9f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:56.412819 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0a97f188-8c08-47d8-9033-c626c86a9f37 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:56.414451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0a97f188-8c08-47d8-9033-c626c86a9f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:56.536312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 367/719] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:56.562794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737725f-783e-4a76-b28e-f35e29556e9b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:56.614179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1efa34-188c-402e-a7ff-22193fb0af7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:56.615268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1efa34-188c-402e-a7ff-22193fb0af7b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:56.615749 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1efa34-188c-402e-a7ff-22193fb0af7b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:56.616216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1efa34-188c-402e-a7ff-22193fb0af7b None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:56.617155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1efa34-188c-402e-a7ff-22193fb0af7b None admin] RBAC: Adding query filter params (name=network) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:56.617771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1efa34-188c-402e-a7ff-22193fb0af7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:56.619440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1efa34-188c-402e-a7ff-22193fb0af7b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:56.628978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 353/720] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:30:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:56.638004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97f188-8c08-47d8-9033-c626c86a9f37 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:56.812504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:56.908785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97f188-8c08-47d8-9033-c626c86a9f37 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:56.999254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff00ca-6032-4248-9193-a74bc9f6769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:57.000301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff00ca-6032-4248-9193-a74bc9f6769c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:57.001286 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff00ca-6032-4248-9193-a74bc9f6769c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:57.001996 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff00ca-6032-4248-9193-a74bc9f6769c None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:57.003268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff00ca-6032-4248-9193-a74bc9f6769c None admin] RBAC: Adding query filter params (type=network) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:57.003804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff00ca-6032-4248-9193-a74bc9f6769c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:57.006594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff00ca-6032-4248-9193-a74bc9f6769c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:57.021804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 368/721] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:30:56 2022] GET /identity/v3/services?type=network => generated 124 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:57.225633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1efa34-188c-402e-a7ff-22193fb0af7b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:57.313216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:57.440013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b2548-51b8-47b7-b492-f697a70fa175 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:57.440537 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b2548-51b8-47b7-b492-f697a70fa175 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:57.440537 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b2548-51b8-47b7-b492-f697a70fa175 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:57.441712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1efa34-188c-402e-a7ff-22193fb0af7b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:57.442297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 369/722] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:57.447708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a39765-d952-4e66-aad4-46e85e593a90 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:57.448030 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a39765-d952-4e66-aad4-46e85e593a90 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:57.448281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a39765-d952-4e66-aad4-46e85e593a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:57.474499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c7a39765-d952-4e66-aad4-46e85e593a90 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:57.476079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c7a39765-d952-4e66-aad4-46e85e593a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:57.517722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6040519-8f94-48c8-abe7-327667ce1212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:57.519187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6040519-8f94-48c8-abe7-327667ce1212 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:57.519637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6040519-8f94-48c8-abe7-327667ce1212 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:57.520073 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6040519-8f94-48c8-abe7-327667ce1212 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:57.541277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6040519-8f94-48c8-abe7-327667ce1212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-d6040519-8f94-48c8-abe7-327667ce1212 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:57.558740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:57.560855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:57.563664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:57.566616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 370/723] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:57.571666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 354/724] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:57.581736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a39765-d952-4e66-aad4-46e85e593a90 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:57.651268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:57.701398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a39765-d952-4e66-aad4-46e85e593a90 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:57.744620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce2a72-4a9d-41d1-9b44-9cdb9c63d775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:57.745136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce2a72-4a9d-41d1-9b44-9cdb9c63d775 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:57.745414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce2a72-4a9d-41d1-9b44-9cdb9c63d775 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:57.745718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce2a72-4a9d-41d1-9b44-9cdb9c63d775 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:57.746356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce2a72-4a9d-41d1-9b44-9cdb9c63d775 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:57.746697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce2a72-4a9d-41d1-9b44-9cdb9c63d775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:57.748114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce2a72-4a9d-41d1-9b44-9cdb9c63d775 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:57.759469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 371/725] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:30:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:57.770332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6040519-8f94-48c8-abe7-327667ce1212 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:57.974664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:57.982286 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6040519-8f94-48c8-abe7-327667ce1212 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:58.046402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce2a72-4a9d-41d1-9b44-9cdb9c63d775 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:58.082570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504661f2-b6ad-4815-a425-d28cec9a80ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:58.083618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504661f2-b6ad-4815-a425-d28cec9a80ae None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:58.084131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504661f2-b6ad-4815-a425-d28cec9a80ae None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:58.084599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504661f2-b6ad-4815-a425-d28cec9a80ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:58.108822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504661f2-b6ad-4815-a425-d28cec9a80ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:58.111429 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504661f2-b6ad-4815-a425-d28cec9a80ae None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-504661f2-b6ad-4815-a425-d28cec9a80ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:58.138303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:58.139351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:58.140267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:58.140267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:58.140267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:58.150946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 355/726] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:58.156489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504661f2-b6ad-4815-a425-d28cec9a80ae None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:58.186044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce2a72-4a9d-41d1-9b44-9cdb9c63d775 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:58.209939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:58.234515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19020-edbc-4e56-8907-34a97c80d1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:58.236089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19020-edbc-4e56-8907-34a97c80d1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:58.236089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19020-edbc-4e56-8907-34a97c80d1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:58.236348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19020-edbc-4e56-8907-34a97c80d1f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:58.249085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa19020-edbc-4e56-8907-34a97c80d1f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-eaa19020-edbc-4e56-8907-34a97c80d1f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:58.254980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:58.255976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:58.257022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 372/727] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:58.271093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19020-edbc-4e56-8907-34a97c80d1f1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:58.409907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504661f2-b6ad-4815-a425-d28cec9a80ae None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:58.491542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3fcf2-66df-4713-bcce-a60f855954b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:58.492562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3fcf2-66df-4713-bcce-a60f855954b7 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:58.493066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3fcf2-66df-4713-bcce-a60f855954b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:58.493560 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3fcf2-66df-4713-bcce-a60f855954b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:58.494821 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3fcf2-66df-4713-bcce-a60f855954b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:58.495360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3fcf2-66df-4713-bcce-a60f855954b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:58.497889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3fcf2-66df-4713-bcce-a60f855954b7 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:58.501785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19020-edbc-4e56-8907-34a97c80d1f1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:58.541728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da29099-5f8a-4082-a86c-02ab518f0627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:58.542150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 356/728] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:58.542343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da29099-5f8a-4082-a86c-02ab518f0627 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:58.542653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da29099-5f8a-4082-a86c-02ab518f0627 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:58.542923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da29099-5f8a-4082-a86c-02ab518f0627 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:58.543690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da29099-5f8a-4082-a86c-02ab518f0627 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:58.543992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da29099-5f8a-4082-a86c-02ab518f0627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:58.545533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da29099-5f8a-4082-a86c-02ab518f0627 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:58.550823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3fcf2-66df-4713-bcce-a60f855954b7 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:58.556477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 373/729] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:30:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:30:58.574916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da29099-5f8a-4082-a86c-02ab518f0627 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:58.584972 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:58.627750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:58.701287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da29099-5f8a-4082-a86c-02ab518f0627 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:58.740566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301599f-a68f-4397-8be7-b36dc5c56ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:58.741632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301599f-a68f-4397-8be7-b36dc5c56ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:58.742133 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301599f-a68f-4397-8be7-b36dc5c56ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:58.742630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301599f-a68f-4397-8be7-b36dc5c56ea4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:58.755553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301599f-a68f-4397-8be7-b36dc5c56ea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:58.757488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301599f-a68f-4397-8be7-b36dc5c56ea4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c301599f-a68f-4397-8be7-b36dc5c56ea4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:30:58.774817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:58.775946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:30:58.776953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:30:58.776953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:30:58.776953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:58.778725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 374/730] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:58.780234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7024-c9bf-4b3a-bd88-c7eb726dee92 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:58.780681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7024-c9bf-4b3a-bd88-c7eb726dee92 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:58.780999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7024-c9bf-4b3a-bd88-c7eb726dee92 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:58.782812 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 375/731] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:58.792983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585430f-08d3-4c97-b54b-f80e0e35a5cb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:58.793243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585430f-08d3-4c97-b54b-f80e0e35a5cb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:58.793578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585430f-08d3-4c97-b54b-f80e0e35a5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:58.832180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e585430f-08d3-4c97-b54b-f80e0e35a5cb None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:58.833467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e585430f-08d3-4c97-b54b-f80e0e35a5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:58.876598 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3fcf2-66df-4713-bcce-a60f855954b7 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:58.926541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 376/732] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 23:30:58.926541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585430f-08d3-4c97-b54b-f80e0e35a5cb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:58.942788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cddc25-1647-400f-8188-2a1b03c149cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:58.943875 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cddc25-1647-400f-8188-2a1b03c149cb None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:58.944374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cddc25-1647-400f-8188-2a1b03c149cb None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:58.944860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cddc25-1647-400f-8188-2a1b03c149cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:58.959934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cddc25-1647-400f-8188-2a1b03c149cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:58.961105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:58.962716 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cddc25-1647-400f-8188-2a1b03c149cb None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-29cddc25-1647-400f-8188-2a1b03c149cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:58.977978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:58.979038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:58.980259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:58.980259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:58.980259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:58.980259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:58.980259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:58.980259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:58.980259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:58.980259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:58.980259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:30:58.980259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:30:58.980259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 357/733] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:30:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:30:58.981934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e7edf-65bc-4b62-9688-590a8a965727 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:58.981934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e7edf-65bc-4b62-9688-590a8a965727 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:58.981934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e7edf-65bc-4b62-9688-590a8a965727 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:58.984688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 358/734] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:30:58.996415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a5ea6-87fd-4f3e-853f-bd0f64e9d065 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:58.996958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a5ea6-87fd-4f3e-853f-bd0f64e9d065 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:58.997790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a5ea6-87fd-4f3e-853f-bd0f64e9d065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:59.020057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-2b9a5ea6-87fd-4f3e-853f-bd0f64e9d065 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:30:59.021704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2b9a5ea6-87fd-4f3e-853f-bd0f64e9d065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:30:59.096292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585430f-08d3-4c97-b54b-f80e0e35a5cb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:59.117298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:59.133917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 359/735] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:30:59.133917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a5ea6-87fd-4f3e-853f-bd0f64e9d065 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:59.149636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd28ad-0f08-4107-aad0-67cb29c448e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:59.150703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd28ad-0f08-4107-aad0-67cb29c448e6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:59.151212 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd28ad-0f08-4107-aad0-67cb29c448e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:59.151688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd28ad-0f08-4107-aad0-67cb29c448e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:59.152582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd28ad-0f08-4107-aad0-67cb29c448e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:59.153125 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd28ad-0f08-4107-aad0-67cb29c448e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:59.163226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd28ad-0f08-4107-aad0-67cb29c448e6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:59.180955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 377/736] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:59.191260 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd28ad-0f08-4107-aad0-67cb29c448e6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:59.356112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:59.389011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a5ea6-87fd-4f3e-853f-bd0f64e9d065 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:59.393010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd28ad-0f08-4107-aad0-67cb29c448e6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:59.467583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5930ee-369e-4485-b9a8-21db80a6ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:59.468184 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf466ea-bb96-4dc7-a090-32eb4d178bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:59.468579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5930ee-369e-4485-b9a8-21db80a6ad29 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:59.469500 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5930ee-369e-4485-b9a8-21db80a6ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:59.469777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf466ea-bb96-4dc7-a090-32eb4d178bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:59.470276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf466ea-bb96-4dc7-a090-32eb4d178bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:59.470788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5930ee-369e-4485-b9a8-21db80a6ad29 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:59.470887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf466ea-bb96-4dc7-a090-32eb4d178bf4 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:59.472203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf466ea-bb96-4dc7-a090-32eb4d178bf4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:30:59.472691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf466ea-bb96-4dc7-a090-32eb4d178bf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:59.476766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf466ea-bb96-4dc7-a090-32eb4d178bf4 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:59.487692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5930ee-369e-4485-b9a8-21db80a6ad29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:59.490596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5930ee-369e-4485-b9a8-21db80a6ad29 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:59.519631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 360/737] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Aug 29 23:30:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:30:59.519631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf466ea-bb96-4dc7-a090-32eb4d178bf4 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1d5930ee-369e-4485-b9a8-21db80a6ad29 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:30:59.520257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:59.521422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 378/738] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Aug 29 23:30:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:59.523684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5930ee-369e-4485-b9a8-21db80a6ad29 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:59.636471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:59.710363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5930ee-369e-4485-b9a8-21db80a6ad29 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:59.720100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf466ea-bb96-4dc7-a090-32eb4d178bf4 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:30:59.771246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb3c9d-e14e-4d76-a529-7e64793117be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:59.772206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb3c9d-e14e-4d76-a529-7e64793117be None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:59.772655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb3c9d-e14e-4d76-a529-7e64793117be None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:59.773115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb3c9d-e14e-4d76-a529-7e64793117be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:59.780708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2582a-e3d1-4138-aad6-71e9d2a62cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:30:59.782872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2582a-e3d1-4138-aad6-71e9d2a62cce None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:30:59.783209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2582a-e3d1-4138-aad6-71e9d2a62cce None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:30:59.783485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2582a-e3d1-4138-aad6-71e9d2a62cce None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:30:59.784362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2582a-e3d1-4138-aad6-71e9d2a62cce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:30:59.786762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2582a-e3d1-4138-aad6-71e9d2a62cce None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:30:59.794200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb3c9d-e14e-4d76-a529-7e64793117be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-2abb3c9d-e14e-4d76-a529-7e64793117be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:30:59.801288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:30:59.811118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 379/739] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:30:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:30:59.813511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb3c9d-e14e-4d76-a529-7e64793117be None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:30:59.815239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:30:59.885957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 361/740] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:30:59 2022] POST /identity/v3/services => generated 251 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:30:59.886765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2582a-e3d1-4138-aad6-71e9d2a62cce None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:00.002357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb3c9d-e14e-4d76-a529-7e64793117be None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:00.054056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d3a99-3906-4c9e-99a7-3ef4eb804c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:00.055586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d3a99-3906-4c9e-99a7-3ef4eb804c33 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:00.056292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d3a99-3906-4c9e-99a7-3ef4eb804c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:00.056973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d3a99-3906-4c9e-99a7-3ef4eb804c33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:00.080050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2582a-e3d1-4138-aad6-71e9d2a62cce None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:00.105063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:00.131165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1f750-62e1-4e45-954a-550f89c0194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:00.132498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f750-62e1-4e45-954a-550f89c0194f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:00.133049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f750-62e1-4e45-954a-550f89c0194f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:00.133549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f750-62e1-4e45-954a-550f89c0194f None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:00.134771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1f750-62e1-4e45-954a-550f89c0194f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:00.135422 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1f750-62e1-4e45-954a-550f89c0194f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:00.137727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1f750-62e1-4e45-954a-550f89c0194f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:00.177271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 362/741] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Aug 29 23:30:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:00.179941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1f750-62e1-4e45-954a-550f89c0194f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:00.232468 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:00.248419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d3a99-3906-4c9e-99a7-3ef4eb804c33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:00.248419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d3a99-3906-4c9e-99a7-3ef4eb804c33 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:00.394250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1f750-62e1-4e45-954a-550f89c0194f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:00.430791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 380/742] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:30:59 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:31:00.431926 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d3a99-3906-4c9e-99a7-3ef4eb804c33 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:00.447741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b403d-aec0-4d92-8df9-bca395ae14ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:00.448383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b403d-aec0-4d92-8df9-bca395ae14ce None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:00.448656 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b403d-aec0-4d92-8df9-bca395ae14ce None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:00.448934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b403d-aec0-4d92-8df9-bca395ae14ce None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:00.449633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b403d-aec0-4d92-8df9-bca395ae14ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:00.449936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b403d-aec0-4d92-8df9-bca395ae14ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:00.451383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b403d-aec0-4d92-8df9-bca395ae14ce None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:00.464270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 363/743] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:31:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:00.471411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb39219-2d5e-4cf3-9961-49b2f0e1c7ab None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:00.471701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb39219-2d5e-4cf3-9961-49b2f0e1c7ab None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:00.472035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb39219-2d5e-4cf3-9961-49b2f0e1c7ab None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:00.473719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 364/744] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:00.474510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb39219-2d5e-4cf3-9961-49b2f0e1c7ab None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:00.530652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:00.591918 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:00.598802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d3a99-3906-4c9e-99a7-3ef4eb804c33 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:00.645416 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb39219-2d5e-4cf3-9961-49b2f0e1c7ab None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:00.659385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12456e2-5100-457c-b24c-cdd8a00fb285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:00.660913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12456e2-5100-457c-b24c-cdd8a00fb285 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:00.661290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12456e2-5100-457c-b24c-cdd8a00fb285 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:00.661658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12456e2-5100-457c-b24c-cdd8a00fb285 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:00.693598 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e98585-4bab-4d86-a585-bad302c59774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:00.694372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98585-4bab-4d86-a585-bad302c59774 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:00.694751 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98585-4bab-4d86-a585-bad302c59774 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:00.695091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98585-4bab-4d86-a585-bad302c59774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:00.709529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e98585-4bab-4d86-a585-bad302c59774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:00.711161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e98585-4bab-4d86-a585-bad302c59774 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-48e98585-4bab-4d86-a585-bad302c59774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:00.728709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:00.729740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:00.730785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:00.730785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:00.730785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:00.734899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 365/745] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:00.735855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e98585-4bab-4d86-a585-bad302c59774 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:00.751598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12456e2-5100-457c-b24c-cdd8a00fb285 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:00.753091 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12456e2-5100-457c-b24c-cdd8a00fb285 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:00.866811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:00.884892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e98585-4bab-4d86-a585-bad302c59774 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:00.896162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:00.906928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 381/746] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:31:00 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 23:31:00.906928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cbf1d-05d9-4e42-9173-1618d0e5a858 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:00.906928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cbf1d-05d9-4e42-9173-1618d0e5a858 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:00.906928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cbf1d-05d9-4e42-9173-1618d0e5a858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:00.927867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-551cbf1d-05d9-4e42-9173-1618d0e5a858 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:00.930004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-551cbf1d-05d9-4e42-9173-1618d0e5a858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:00.945739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909aad8-f266-4f31-b4c0-2471d08d4197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:00.946778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909aad8-f266-4f31-b4c0-2471d08d4197 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:00.947249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909aad8-f266-4f31-b4c0-2471d08d4197 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:00.947753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909aad8-f266-4f31-b4c0-2471d08d4197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:00.966726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909aad8-f266-4f31-b4c0-2471d08d4197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:00.969170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909aad8-f266-4f31-b4c0-2471d08d4197 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-f909aad8-f266-4f31-b4c0-2471d08d4197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:00.993615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:00.995759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:00.997528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:00.997528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:00.997528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:00.997528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 366/747] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:00.997528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909aad8-f266-4f31-b4c0-2471d08d4197 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:01.070789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 382/748] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 23:31:01.072570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cbf1d-05d9-4e42-9173-1618d0e5a858 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:01.214133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909aad8-f266-4f31-b4c0-2471d08d4197 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:01.303388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cbf1d-05d9-4e42-9173-1618d0e5a858 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:01.312788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e3b29-6c94-499d-883d-8fe3b2a56726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:01.314077 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e3b29-6c94-499d-883d-8fe3b2a56726 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:01.314356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e3b29-6c94-499d-883d-8fe3b2a56726 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:01.314893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e3b29-6c94-499d-883d-8fe3b2a56726 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:01.316127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e3b29-6c94-499d-883d-8fe3b2a56726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:01.316680 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e3b29-6c94-499d-883d-8fe3b2a56726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:01.323736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e3b29-6c94-499d-883d-8fe3b2a56726 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:01.372089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 367/749] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:01.374195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e3b29-6c94-499d-883d-8fe3b2a56726 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:01.382172 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:01.395260 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:01.405496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9420864-99d5-4018-940c-21ac168f7ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:01.406154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9420864-99d5-4018-940c-21ac168f7ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:01.407334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9420864-99d5-4018-940c-21ac168f7ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:01.407675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9420864-99d5-4018-940c-21ac168f7ca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:01.418979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9420864-99d5-4018-940c-21ac168f7ca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:01.420478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9420864-99d5-4018-940c-21ac168f7ca9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e9420864-99d5-4018-940c-21ac168f7ca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:01.440601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:01.442505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:01.444188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:01.444188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:01.444188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:01.444188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 383/750] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:01.444188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9420864-99d5-4018-940c-21ac168f7ca9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:01.570766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e3b29-6c94-499d-883d-8fe3b2a56726 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:01.596675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9420864-99d5-4018-940c-21ac168f7ca9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:01.638889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdcf15-466d-4dbe-a9a8-4ce8047b521d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:01.639895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdcf15-466d-4dbe-a9a8-4ce8047b521d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:01.640386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdcf15-466d-4dbe-a9a8-4ce8047b521d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:01.642314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdcf15-466d-4dbe-a9a8-4ce8047b521d None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:01.644165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdcf15-466d-4dbe-a9a8-4ce8047b521d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:01.644701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdcf15-466d-4dbe-a9a8-4ce8047b521d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:01.647342 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdcf15-466d-4dbe-a9a8-4ce8047b521d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:01.653684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea321477-f0c0-4ddc-a5f0-1027c31e63ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:01.654517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321477-f0c0-4ddc-a5f0-1027c31e63ca None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:01.654924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321477-f0c0-4ddc-a5f0-1027c31e63ca None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:01.655336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321477-f0c0-4ddc-a5f0-1027c31e63ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:01.666984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea321477-f0c0-4ddc-a5f0-1027c31e63ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:01.668502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea321477-f0c0-4ddc-a5f0-1027c31e63ca None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:01.677827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 368/751] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:01.677827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdcf15-466d-4dbe-a9a8-4ce8047b521d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ea321477-f0c0-4ddc-a5f0-1027c31e63ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:01.689804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:01.691830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:01.693605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:01.693605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:01.693605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:01.693605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:01.693605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:01.693605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:01.693605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:01.693605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:01.693605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 384/752] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:01.693605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea321477-f0c0-4ddc-a5f0-1027c31e63ca None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:01.784413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:01.794028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:01.854923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea321477-f0c0-4ddc-a5f0-1027c31e63ca None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:01.880475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdcf15-466d-4dbe-a9a8-4ce8047b521d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:01.901394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5e0ab-f9fc-411f-8d06-062092d98881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:01.902024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5e0ab-f9fc-411f-8d06-062092d98881 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:01.902301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5e0ab-f9fc-411f-8d06-062092d98881 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:01.902605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5e0ab-f9fc-411f-8d06-062092d98881 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:01.903279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5e0ab-f9fc-411f-8d06-062092d98881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:01.903561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5e0ab-f9fc-411f-8d06-062092d98881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:01.904966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5e0ab-f9fc-411f-8d06-062092d98881 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:01.919760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 385/753] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:01.925547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5e0ab-f9fc-411f-8d06-062092d98881 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:01.942296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee2b3f-c1de-4dcc-94f5-fcd1462d767d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:01.942955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2b3f-c1de-4dcc-94f5-fcd1462d767d None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:01.943234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2b3f-c1de-4dcc-94f5-fcd1462d767d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:01.943488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2b3f-c1de-4dcc-94f5-fcd1462d767d None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:01.944157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee2b3f-c1de-4dcc-94f5-fcd1462d767d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:01.945537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee2b3f-c1de-4dcc-94f5-fcd1462d767d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:02.029308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 369/754] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:31:01 2022] POST /identity/v3/services => generated 240 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:31:02.032326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee2b3f-c1de-4dcc-94f5-fcd1462d767d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:02.129316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5e0ab-f9fc-411f-8d06-062092d98881 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:02.132533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:02.218338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d6a3c-b91c-4c76-951b-d513f69c1f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:02.221800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d6a3c-b91c-4c76-951b-d513f69c1f1a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:02.222312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d6a3c-b91c-4c76-951b-d513f69c1f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:02.222855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d6a3c-b91c-4c76-951b-d513f69c1f1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:02.241462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d6a3c-b91c-4c76-951b-d513f69c1f1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:02.244410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d6a3c-b91c-4c76-951b-d513f69c1f1a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e81d6a3c-b91c-4c76-951b-d513f69c1f1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:02.270294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:02.272167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:02.273853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:02.273853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:02.273853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:02.273853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:02.273853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:02.273853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:02.273853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:02.273853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:02.273853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:31:02.273853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:02.281953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 386/755] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Aug 29 23:31:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:02.281953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d6a3c-b91c-4c76-951b-d513f69c1f1a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:02.282631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee2b3f-c1de-4dcc-94f5-fcd1462d767d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:02.341944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:02.375357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafce85-e8cd-4f74-8d2c-8bdc6e0b30cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:02.376521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafce85-e8cd-4f74-8d2c-8bdc6e0b30cc None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:02.376673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafce85-e8cd-4f74-8d2c-8bdc6e0b30cc None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:02.376886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafce85-e8cd-4f74-8d2c-8bdc6e0b30cc None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:02.378466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafce85-e8cd-4f74-8d2c-8bdc6e0b30cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:02.379072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafce85-e8cd-4f74-8d2c-8bdc6e0b30cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:02.380601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafce85-e8cd-4f74-8d2c-8bdc6e0b30cc None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:02.417278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 370/756] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:02.419390 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafce85-e8cd-4f74-8d2c-8bdc6e0b30cc None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:02.511222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d6a3c-b91c-4c76-951b-d513f69c1f1a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:02.553709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafce85-e8cd-4f74-8d2c-8bdc6e0b30cc None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:02.586861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe023e-0480-4f1e-84ad-5af0b8e7987f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:02.587914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe023e-0480-4f1e-84ad-5af0b8e7987f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:02.588403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe023e-0480-4f1e-84ad-5af0b8e7987f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:02.588913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe023e-0480-4f1e-84ad-5af0b8e7987f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:02.598570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe023e-0480-4f1e-84ad-5af0b8e7987f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:02.600845 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe023e-0480-4f1e-84ad-5af0b8e7987f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-59fe023e-0480-4f1e-84ad-5af0b8e7987f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:02.616319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:02.617352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:02.618241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:02.618241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:02.618241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:02.618241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:02.618241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:02.618241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:02.618241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:02.618241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:02.624781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 387/757] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:02.624781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe023e-0480-4f1e-84ad-5af0b8e7987f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:02.625153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f441e9-b954-4f0f-8318-3af0f614e600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:02.625153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f441e9-b954-4f0f-8318-3af0f614e600 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:02.625153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f441e9-b954-4f0f-8318-3af0f614e600 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:02.625153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f441e9-b954-4f0f-8318-3af0f614e600 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:02.625752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f441e9-b954-4f0f-8318-3af0f614e600 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:02.626235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f441e9-b954-4f0f-8318-3af0f614e600 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:02.629048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f441e9-b954-4f0f-8318-3af0f614e600 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:02.661291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:02.669274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 371/758] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Aug 29 23:31:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:02.669274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188dbfc-6ece-477a-8ed1-070d8fa00c68 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:02.669274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188dbfc-6ece-477a-8ed1-070d8fa00c68 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:02.669274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188dbfc-6ece-477a-8ed1-070d8fa00c68 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:02.673900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 372/759] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:02.673900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188dbfc-6ece-477a-8ed1-070d8fa00c68 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:02.749706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:02.818071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe023e-0480-4f1e-84ad-5af0b8e7987f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:02.840827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188dbfc-6ece-477a-8ed1-070d8fa00c68 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:02.860871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066b5d6-a745-40b6-9c61-b66d37d29896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:02.861515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066b5d6-a745-40b6-9c61-b66d37d29896 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:02.861791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066b5d6-a745-40b6-9c61-b66d37d29896 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:02.862065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066b5d6-a745-40b6-9c61-b66d37d29896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:02.862877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6066b5d6-a745-40b6-9c61-b66d37d29896 None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=c6b37b09a1ae45e8bc16adfa59500eb9) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:02.863204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6066b5d6-a745-40b6-9c61-b66d37d29896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:02.864720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6066b5d6-a745-40b6-9c61-b66d37d29896 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:02.878460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 388/760] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:31:02 2022] GET /identity/v3/role_assignments?user.id=c6b37b09a1ae45e8bc16adfa59500eb9&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=d86dba8efe20423abdde06031690beb8 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:02.878460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066b5d6-a745-40b6-9c61-b66d37d29896 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:02.921754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9163b2-aab5-4082-ad17-b4323f2561bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:02.922961 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9163b2-aab5-4082-ad17-b4323f2561bb None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:02.923396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9163b2-aab5-4082-ad17-b4323f2561bb None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:02.923841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9163b2-aab5-4082-ad17-b4323f2561bb None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:02.925006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9163b2-aab5-4082-ad17-b4323f2561bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:02.925625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9163b2-aab5-4082-ad17-b4323f2561bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:02.928648 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9163b2-aab5-4082-ad17-b4323f2561bb None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:02.952750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 373/761] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:02.955633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd79b0e-4c02-4e42-a201-54856716fb10 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:02.956077 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd79b0e-4c02-4e42-a201-54856716fb10 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:02.956523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd79b0e-4c02-4e42-a201-54856716fb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:02.992115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-9dd79b0e-4c02-4e42-a201-54856716fb10 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:02.994186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9dd79b0e-4c02-4e42-a201-54856716fb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:03.071474 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:03.103507 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066b5d6-a745-40b6-9c61-b66d37d29896 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:03.150906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 374/762] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:03.152360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd79b0e-4c02-4e42-a201-54856716fb10 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:03.179578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b73644-529f-4d95-b313-306a5ce5ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:03.180514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b73644-529f-4d95-b313-306a5ce5ffda None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:03.180975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b73644-529f-4d95-b313-306a5ce5ffda None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:03.181439 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b73644-529f-4d95-b313-306a5ce5ffda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:03.220602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b73644-529f-4d95-b313-306a5ce5ffda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:03.223188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b73644-529f-4d95-b313-306a5ce5ffda None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:03.268589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d5b73644-529f-4d95-b313-306a5ce5ffda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:03.269636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:03.270931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:03.272603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:03.272603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:03.272603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:03.273964 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 389/763] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:03.283194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b73644-529f-4d95-b313-306a5ce5ffda None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:03.522229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd79b0e-4c02-4e42-a201-54856716fb10 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:03.636603 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499b9fe-0994-4442-9844-b2ff3431c7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:03.637242 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b9fe-0994-4442-9844-b2ff3431c7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:03.637593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b9fe-0994-4442-9844-b2ff3431c7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:03.637870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b9fe-0994-4442-9844-b2ff3431c7d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:03.638644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499b9fe-0994-4442-9844-b2ff3431c7d7 None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=ff5b4f2f41604a8286141bce23db35c5) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:03.638859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b73644-529f-4d95-b313-306a5ce5ffda None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:03.639228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499b9fe-0994-4442-9844-b2ff3431c7d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:03.640545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499b9fe-0994-4442-9844-b2ff3431c7d7 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:03.653978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 375/764] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:31:03 2022] GET /identity/v3/role_assignments?user.id=ff5b4f2f41604a8286141bce23db35c5&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 259 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:03.658452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499b9fe-0994-4442-9844-b2ff3431c7d7 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:03.759959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb5017-8a10-4236-ada7-fe63093a5b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:03.760590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb5017-8a10-4236-ada7-fe63093a5b0d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:03.760870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb5017-8a10-4236-ada7-fe63093a5b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:03.761696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb5017-8a10-4236-ada7-fe63093a5b0d None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:03.762485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb5017-8a10-4236-ada7-fe63093a5b0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:03.762800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb5017-8a10-4236-ada7-fe63093a5b0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:03.766243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb5017-8a10-4236-ada7-fe63093a5b0d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:03.768861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:03.796293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:03.800183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 390/765] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 519 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:03.808346 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb5017-8a10-4236-ada7-fe63093a5b0d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:03.809880 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499b9fe-0994-4442-9844-b2ff3431c7d7 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:03.849806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb20d0a-ed1f-43d0-bc5a-84863fe9e416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:03.850587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb20d0a-ed1f-43d0-bc5a-84863fe9e416 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:03.850953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb20d0a-ed1f-43d0-bc5a-84863fe9e416 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:03.851294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb20d0a-ed1f-43d0-bc5a-84863fe9e416 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:03.851980 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb20d0a-ed1f-43d0-bc5a-84863fe9e416 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:03.853476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb20d0a-ed1f-43d0-bc5a-84863fe9e416 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-fcb20d0a-ed1f-43d0-bc5a-84863fe9e416 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:03.864146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:03.865503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 23:31:03.866575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:03.870700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 376/766] 10.222.0.58 () {56 vars in 1129 bytes} [Mon Aug 29 23:31:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:03.874136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb20d0a-ed1f-43d0-bc5a-84863fe9e416 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:03.992251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb20d0a-ed1f-43d0-bc5a-84863fe9e416 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:04.036694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad94944-11bb-4e17-966b-2b576b56f616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:04.037726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad94944-11bb-4e17-966b-2b576b56f616 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:04.038228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad94944-11bb-4e17-966b-2b576b56f616 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:04.038748 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad94944-11bb-4e17-966b-2b576b56f616 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:04.040081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad94944-11bb-4e17-966b-2b576b56f616 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:04.041485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad94944-11bb-4e17-966b-2b576b56f616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:04.042960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad94944-11bb-4e17-966b-2b576b56f616 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:04.043827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb5017-8a10-4236-ada7-fe63093a5b0d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:04.051554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 377/767] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Aug 29 23:31:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:04.061748 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad94944-11bb-4e17-966b-2b576b56f616 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:04.097750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:04.143455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000d6b6-ed17-477b-93f5-e083dd5afda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:04.144416 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000d6b6-ed17-477b-93f5-e083dd5afda8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:04.144846 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000d6b6-ed17-477b-93f5-e083dd5afda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:04.145292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000d6b6-ed17-477b-93f5-e083dd5afda8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:04.168253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000d6b6-ed17-477b-93f5-e083dd5afda8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:04.170092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000d6b6-ed17-477b-93f5-e083dd5afda8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:04.185521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad94944-11bb-4e17-966b-2b576b56f616 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-0000d6b6-ed17-477b-93f5-e083dd5afda8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:04.203589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:04.204939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:04.206019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:04.206019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:04.206019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:04.206019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:04.206019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:04.206019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:04.206019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:04.206019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:04.218614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 391/768] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:04.223522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000d6b6-ed17-477b-93f5-e083dd5afda8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:04.227017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c386579-b6ed-4839-8b8d-ad1c95f4c9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:04.227734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c386579-b6ed-4839-8b8d-ad1c95f4c9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:04.228155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c386579-b6ed-4839-8b8d-ad1c95f4c9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:04.228498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c386579-b6ed-4839-8b8d-ad1c95f4c9e6 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:04.229230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c386579-b6ed-4839-8b8d-ad1c95f4c9e6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:04.229603 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c386579-b6ed-4839-8b8d-ad1c95f4c9e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:04.231160 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c386579-b6ed-4839-8b8d-ad1c95f4c9e6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:04.237126 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 378/769] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Aug 29 23:31:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:04.243321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c386579-b6ed-4839-8b8d-ad1c95f4c9e6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:04.355468 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:04.428674 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000d6b6-ed17-477b-93f5-e083dd5afda8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:04.481798 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:04.494919 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c386579-b6ed-4839-8b8d-ad1c95f4c9e6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:04.535092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71515d-63b1-40ef-b181-0ea9abad0d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:04.535737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71515d-63b1-40ef-b181-0ea9abad0d87 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:04.536021 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71515d-63b1-40ef-b181-0ea9abad0d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:04.536796 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71515d-63b1-40ef-b181-0ea9abad0d87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:04.537591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71515d-63b1-40ef-b181-0ea9abad0d87 None admin] RBAC: Adding query filter params (interface=public, service_id=ca8bb630bf05405d9993dadc4959c535, region_id=RegionOne) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:04.537911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71515d-63b1-40ef-b181-0ea9abad0d87 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:04.539375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71515d-63b1-40ef-b181-0ea9abad0d87 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:04.550632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 379/770] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Aug 29 23:31:04 2022] GET /identity/v3/endpoints?service_id=ca8bb630bf05405d9993dadc4959c535&interface=public®ion_id=RegionOne => generated 194 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:04.556777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71515d-63b1-40ef-b181-0ea9abad0d87 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:04.593411 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f590abc-bae5-4fb0-bee7-49c726444163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:04.594894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f590abc-bae5-4fb0-bee7-49c726444163 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:04.594894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f590abc-bae5-4fb0-bee7-49c726444163 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:04.594894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f590abc-bae5-4fb0-bee7-49c726444163 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:04.598451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f590abc-bae5-4fb0-bee7-49c726444163 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:04.598451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f590abc-bae5-4fb0-bee7-49c726444163 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:04.598451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f590abc-bae5-4fb0-bee7-49c726444163 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:04.613790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 392/771] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:04.622519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f590abc-bae5-4fb0-bee7-49c726444163 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:04.802784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71515d-63b1-40ef-b181-0ea9abad0d87 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:04.880954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8e54c-c3ae-4afd-a577-27caaa423d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:04.881868 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8e54c-c3ae-4afd-a577-27caaa423d5f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:04.882368 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8e54c-c3ae-4afd-a577-27caaa423d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:04.882880 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8e54c-c3ae-4afd-a577-27caaa423d5f None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:04.883972 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8e54c-c3ae-4afd-a577-27caaa423d5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:04.887417 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8e54c-c3ae-4afd-a577-27caaa423d5f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:04.900293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 380/772] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Aug 29 23:31:04 2022] GET /identity/v3/services => generated 772 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:04.961635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f590abc-bae5-4fb0-bee7-49c726444163 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:04.987084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:05.015534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6a009-15d2-40c3-b9ec-25e075bad754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:05.016997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a009-15d2-40c3-b9ec-25e075bad754 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:05.016997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a009-15d2-40c3-b9ec-25e075bad754 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:05.016997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a009-15d2-40c3-b9ec-25e075bad754 None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/e4a8261d92064b838fc6e8f415879566/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:05.019017 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6a009-15d2-40c3-b9ec-25e075bad754 None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=e4a8261d92064b838fc6e8f415879566, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:05.020553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6a009-15d2-40c3-b9ec-25e075bad754 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:05.101765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 393/773] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:31:04 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/e4a8261d92064b838fc6e8f415879566/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:31:05.470752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad930d02-86d3-4fcd-916e-1e129ed7eb84 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:05.471015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad930d02-86d3-4fcd-916e-1e129ed7eb84 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:05.471458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad930d02-86d3-4fcd-916e-1e129ed7eb84 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:05.473014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 381/774] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:05.480815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e3b4-b168-449e-9c11-29f7165a972e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:05.481206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e3b4-b168-449e-9c11-29f7165a972e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:05.481412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e3b4-b168-449e-9c11-29f7165a972e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:05.512504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-09d5e3b4-b168-449e-9c11-29f7165a972e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:05.513723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-09d5e3b4-b168-449e-9c11-29f7165a972e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:05.611804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 394/775] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:06.173086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad930d02-86d3-4fcd-916e-1e129ed7eb84 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:06.208631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:06.233238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903b745-0f8e-4288-96e0-e5b0a7f5cfe8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:06.234646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903b745-0f8e-4288-96e0-e5b0a7f5cfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:06.234646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903b745-0f8e-4288-96e0-e5b0a7f5cfe8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:06.237390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 395/776] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:06.243792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d003a5-e32a-44d6-879a-7eedc88a224a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:06.244259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d003a5-e32a-44d6-879a-7eedc88a224a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:06.244728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d003a5-e32a-44d6-879a-7eedc88a224a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:06.299801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b1d003a5-e32a-44d6-879a-7eedc88a224a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:06.301883 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b1d003a5-e32a-44d6-879a-7eedc88a224a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:06.413035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad930d02-86d3-4fcd-916e-1e129ed7eb84 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:06.496690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adf669-92ec-4999-962a-000ba71dc596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:06.497622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adf669-92ec-4999-962a-000ba71dc596 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:06.498084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adf669-92ec-4999-962a-000ba71dc596 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:06.498565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adf669-92ec-4999-962a-000ba71dc596 None admin] PATH_INFO: `/v3/services/network` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:06.499601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19adf669-92ec-4999-962a-000ba71dc596 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:06.502057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19adf669-92ec-4999-962a-000ba71dc596 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:06.508622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 396/777] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-19adf669-92ec-4999-962a-000ba71dc596 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:06.539781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:06.542006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:06.544045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 382/778] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:06 2022] GET /identity/v3/services/network => generated 88 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:06.557044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d003a5-e32a-44d6-879a-7eedc88a224a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:06.642285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:06.754827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d003a5-e32a-44d6-879a-7eedc88a224a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:06.837051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecda9d3-8c4c-4c4a-b76f-5c2740917f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:06.838093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecda9d3-8c4c-4c4a-b76f-5c2740917f13 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:06.838619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecda9d3-8c4c-4c4a-b76f-5c2740917f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:06.839105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecda9d3-8c4c-4c4a-b76f-5c2740917f13 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:06.840293 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecda9d3-8c4c-4c4a-b76f-5c2740917f13 None admin] RBAC: Adding query filter params (name=network) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:06.840816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecda9d3-8c4c-4c4a-b76f-5c2740917f13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:06.843997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecda9d3-8c4c-4c4a-b76f-5c2740917f13 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:06.853185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 397/779] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:31:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:06.863304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adf669-92ec-4999-962a-000ba71dc596 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:06.883755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecda9d3-8c4c-4c4a-b76f-5c2740917f13 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:07.010959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:07.042024 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adf669-92ec-4999-962a-000ba71dc596 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:07.133096 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae36f46-c65f-4319-8080-d3fa9d326342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:07.134208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae36f46-c65f-4319-8080-d3fa9d326342 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:07.134740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae36f46-c65f-4319-8080-d3fa9d326342 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:07.135237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae36f46-c65f-4319-8080-d3fa9d326342 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:07.136520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae36f46-c65f-4319-8080-d3fa9d326342 None admin] RBAC: Adding query filter params (type=network) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:07.137071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae36f46-c65f-4319-8080-d3fa9d326342 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:07.139682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae36f46-c65f-4319-8080-d3fa9d326342 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:07.143552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:07.152916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecda9d3-8c4c-4c4a-b76f-5c2740917f13 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:07.166513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 383/780] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:31:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:07.175928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae36f46-c65f-4319-8080-d3fa9d326342 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:07.219177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef1bdf-d062-491c-ac72-2b15a7bbe063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:07.219829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef1bdf-d062-491c-ac72-2b15a7bbe063 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:07.220105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef1bdf-d062-491c-ac72-2b15a7bbe063 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:07.220444 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef1bdf-d062-491c-ac72-2b15a7bbe063 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:07.232698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef1bdf-d062-491c-ac72-2b15a7bbe063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-bcef1bdf-d062-491c-ac72-2b15a7bbe063 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:07.238257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:07.241538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:07.243240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 398/781] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:31:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:07.245770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef1bdf-d062-491c-ac72-2b15a7bbe063 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:07.362026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef1bdf-d062-491c-ac72-2b15a7bbe063 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:07.404577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cffa14-9563-477f-9279-b4cf40daa71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:07.405194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cffa14-9563-477f-9279-b4cf40daa71f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:07.405475 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cffa14-9563-477f-9279-b4cf40daa71f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:07.406305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cffa14-9563-477f-9279-b4cf40daa71f None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:07.408167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cffa14-9563-477f-9279-b4cf40daa71f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:07.408492 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cffa14-9563-477f-9279-b4cf40daa71f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:07.409913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cffa14-9563-477f-9279-b4cf40daa71f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:07.431791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 399/782] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:31:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:07.433475 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cffa14-9563-477f-9279-b4cf40daa71f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:07.464987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae36f46-c65f-4319-8080-d3fa9d326342 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:07.478273 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:07.523674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88bb52-e500-4862-9983-4167575cc715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:07.524658 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88bb52-e500-4862-9983-4167575cc715 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:07.525165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88bb52-e500-4862-9983-4167575cc715 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:07.525657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88bb52-e500-4862-9983-4167575cc715 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:07.526639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88bb52-e500-4862-9983-4167575cc715 None admin] RBAC: Adding query filter params (interface=public, service_id=07ef52b6a9cd45e2b80389f6555e5c69, region_id=RegionOne) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:07.527183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88bb52-e500-4862-9983-4167575cc715 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:07.529699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88bb52-e500-4862-9983-4167575cc715 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:07.538863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 384/783] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Aug 29 23:31:07 2022] GET /identity/v3/endpoints?service_id=07ef52b6a9cd45e2b80389f6555e5c69&interface=public®ion_id=RegionOne => generated 194 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:07.540794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0b959-bdb4-49ad-829c-3dc442c8e46f None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:07.541538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0b959-bdb4-49ad-829c-3dc442c8e46f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:07.541963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0b959-bdb4-49ad-829c-3dc442c8e46f None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:07.544478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 385/784] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:07.548319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0b959-bdb4-49ad-829c-3dc442c8e46f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:07.582981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:07.635675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cffa14-9563-477f-9279-b4cf40daa71f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:07.756646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf88cdc-9ca0-4b7e-8461-0ed4bc48e8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:07.758024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf88cdc-9ca0-4b7e-8461-0ed4bc48e8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:07.758653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf88cdc-9ca0-4b7e-8461-0ed4bc48e8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:07.759252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf88cdc-9ca0-4b7e-8461-0ed4bc48e8d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:07.793083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf88cdc-9ca0-4b7e-8461-0ed4bc48e8d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:07.796061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf88cdc-9ca0-4b7e-8461-0ed4bc48e8d1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ddf88cdc-9ca0-4b7e-8461-0ed4bc48e8d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:07.851884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:07.853572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:07.855045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:07.855045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:07.855045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:07.862391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 400/785] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:07.870458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07676278-ca5a-4675-9d80-9224dab04a6a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:07.870458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07676278-ca5a-4675-9d80-9224dab04a6a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:07.870458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07676278-ca5a-4675-9d80-9224dab04a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:07.878459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0b959-bdb4-49ad-829c-3dc442c8e46f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:07.946957 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-07676278-ca5a-4675-9d80-9224dab04a6a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:07.949725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-07676278-ca5a-4675-9d80-9224dab04a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:07.964095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac071d-866f-401e-9a2e-d1d474ba7f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:07.965193 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac071d-866f-401e-9a2e-d1d474ba7f00 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:07.965676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac071d-866f-401e-9a2e-d1d474ba7f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:07.968455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac071d-866f-401e-9a2e-d1d474ba7f00 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:07.973370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac071d-866f-401e-9a2e-d1d474ba7f00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:07.976277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac071d-866f-401e-9a2e-d1d474ba7f00 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:08.000946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 386/786] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Aug 29 23:31:07 2022] GET /identity/v3/services => generated 772 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:08.002611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac071d-866f-401e-9a2e-d1d474ba7f00 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:08.195548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 401/787] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:08.198547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24b659-8e4a-4a9e-9b2c-1250f86ba7c9 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:08.198547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24b659-8e4a-4a9e-9b2c-1250f86ba7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:08.205591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24b659-8e4a-4a9e-9b2c-1250f86ba7c9 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:08.208242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 402/788] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:08.219746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6fa78-f22d-4811-a315-7900ffad4184 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:08.220231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6fa78-f22d-4811-a315-7900ffad4184 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:08.220683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6fa78-f22d-4811-a315-7900ffad4184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:08.236135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:08.262457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:08.334457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5eb6fa78-f22d-4811-a315-7900ffad4184 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:08.334457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5eb6fa78-f22d-4811-a315-7900ffad4184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:08.548340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 403/789] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:08.552128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b7bb8-2ba7-432f-a9c8-c0a40348b806 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:08.552706 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b7bb8-2ba7-432f-a9c8-c0a40348b806 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:08.553207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b7bb8-2ba7-432f-a9c8-c0a40348b806 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:08.556430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 404/790] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:08.566154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52424-a692-4f15-a8d1-c677f0c5db26 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:08.566747 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52424-a692-4f15-a8d1-c677f0c5db26 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:08.567387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52424-a692-4f15-a8d1-c677f0c5db26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:08.592539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac071d-866f-401e-9a2e-d1d474ba7f00 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:08.618759 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d3d52424-a692-4f15-a8d1-c677f0c5db26 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:08.620439 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d3d52424-a692-4f15-a8d1-c677f0c5db26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:08.661765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e842-b90b-4282-b6d7-a7d08e5a673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:08.662996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e842-b90b-4282-b6d7-a7d08e5a673e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:08.663570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e842-b90b-4282-b6d7-a7d08e5a673e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:08.664056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e842-b90b-4282-b6d7-a7d08e5a673e None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:08.665198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9e842-b90b-4282-b6d7-a7d08e5a673e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:08.665691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9e842-b90b-4282-b6d7-a7d08e5a673e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:08.668270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9e842-b90b-4282-b6d7-a7d08e5a673e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:08.687512 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 387/791] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 686 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:08.690557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e842-b90b-4282-b6d7-a7d08e5a673e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:08.759294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 405/792] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:08.761007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d52424-a692-4f15-a8d1-c677f0c5db26 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:08.919439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e842-b90b-4282-b6d7-a7d08e5a673e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:08.946652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:08.954820 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:08.966475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41de5c1-d9fe-46de-a8a1-82c7a7cc057f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:08.967540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41de5c1-d9fe-46de-a8a1-82c7a7cc057f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:08.967869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41de5c1-d9fe-46de-a8a1-82c7a7cc057f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:08.968873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41de5c1-d9fe-46de-a8a1-82c7a7cc057f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:08.998951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41de5c1-d9fe-46de-a8a1-82c7a7cc057f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:09.006333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d52424-a692-4f15-a8d1-c677f0c5db26 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-e41de5c1-d9fe-46de-a8a1-82c7a7cc057f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:09.020104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:09.021395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:31:09.022347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:09.023087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 388/793] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:31:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:09.023562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41de5c1-d9fe-46de-a8a1-82c7a7cc057f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:09.060020 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39690e-22f8-407b-9172-b50d8b29464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:09.061400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39690e-22f8-407b-9172-b50d8b29464c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:09.062509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39690e-22f8-407b-9172-b50d8b29464c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:09.063023 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39690e-22f8-407b-9172-b50d8b29464c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:09.073172 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39690e-22f8-407b-9172-b50d8b29464c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:09.075388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39690e-22f8-407b-9172-b50d8b29464c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-dd39690e-22f8-407b-9172-b50d8b29464c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:09.091707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:09.092763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:09.093756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:09.093756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:09.093756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:09.093756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:09.093756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:09.093756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:09.093756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:09.093756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:09.093756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:31:09.093756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:09.102506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 406/794] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:31:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:09.103889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39690e-22f8-407b-9172-b50d8b29464c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:09.196577 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41de5c1-d9fe-46de-a8a1-82c7a7cc057f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:09.261188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39690e-22f8-407b-9172-b50d8b29464c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:09.264104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59ed3e-f86c-4cba-99bb-c14b86908c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:09.267032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ed3e-f86c-4cba-99bb-c14b86908c9e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:09.267032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ed3e-f86c-4cba-99bb-c14b86908c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:09.267266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ed3e-f86c-4cba-99bb-c14b86908c9e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:09.268449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59ed3e-f86c-4cba-99bb-c14b86908c9e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:09.271376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59ed3e-f86c-4cba-99bb-c14b86908c9e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-6b59ed3e-f86c-4cba-99bb-c14b86908c9e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:09.289494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:09.290645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 23:31:09.291521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:09.295392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:09.297802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 389/795] 10.222.0.58 () {56 vars in 1129 bytes} [Mon Aug 29 23:31:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:09.299549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59ed3e-f86c-4cba-99bb-c14b86908c9e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:09.316962 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9f51a-2e37-46c2-b6ac-6b4cd36ef8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:09.317887 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9f51a-2e37-46c2-b6ac-6b4cd36ef8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:09.318168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9f51a-2e37-46c2-b6ac-6b4cd36ef8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:09.318471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9f51a-2e37-46c2-b6ac-6b4cd36ef8d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:09.320758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9f51a-2e37-46c2-b6ac-6b4cd36ef8d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:09.320758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9f51a-2e37-46c2-b6ac-6b4cd36ef8d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:09.324030 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9f51a-2e37-46c2-b6ac-6b4cd36ef8d0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:09.336098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 407/796] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:31:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:09.338052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9f51a-2e37-46c2-b6ac-6b4cd36ef8d0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:09.404298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:09.463735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59ed3e-f86c-4cba-99bb-c14b86908c9e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:09.512557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282609d9-a2c5-4a67-a767-6776b66ffdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:09.514032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282609d9-a2c5-4a67-a767-6776b66ffdb8 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:09.514821 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282609d9-a2c5-4a67-a767-6776b66ffdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:09.516085 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282609d9-a2c5-4a67-a767-6776b66ffdb8 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:09.517990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282609d9-a2c5-4a67-a767-6776b66ffdb8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:09.518307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282609d9-a2c5-4a67-a767-6776b66ffdb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:09.522464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282609d9-a2c5-4a67-a767-6776b66ffdb8 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:09.523025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9f51a-2e37-46c2-b6ac-6b4cd36ef8d0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:09.565029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 390/797] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Aug 29 23:31:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:09.566601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282609d9-a2c5-4a67-a767-6776b66ffdb8 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:09.572247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666964e-dc96-4bfd-a12d-7b9a60102a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:09.573609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666964e-dc96-4bfd-a12d-7b9a60102a12 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:09.574033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666964e-dc96-4bfd-a12d-7b9a60102a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:09.574499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666964e-dc96-4bfd-a12d-7b9a60102a12 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:09.575291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666964e-dc96-4bfd-a12d-7b9a60102a12 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:09.575719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666964e-dc96-4bfd-a12d-7b9a60102a12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:09.577266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666964e-dc96-4bfd-a12d-7b9a60102a12 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:09.583516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 408/798] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Aug 29 23:31:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:09.584546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666964e-dc96-4bfd-a12d-7b9a60102a12 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:09.677126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:09.735917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282609d9-a2c5-4a67-a767-6776b66ffdb8 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:09.759717 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:09.765540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666964e-dc96-4bfd-a12d-7b9a60102a12 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:09.796370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d04c4-c12f-4980-99d5-bdc18807d937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:09.797406 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d04c4-c12f-4980-99d5-bdc18807d937 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:09.797682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d04c4-c12f-4980-99d5-bdc18807d937 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:09.797950 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d04c4-c12f-4980-99d5-bdc18807d937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:09.810716 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d04c4-c12f-4980-99d5-bdc18807d937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:09.821760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b331550-6f3b-48a5-9cbf-38109953b55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-986d04c4-c12f-4980-99d5-bdc18807d937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:09.822382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:09.824273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:31:09.826016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:09.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b331550-6f3b-48a5-9cbf-38109953b55a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:09.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b331550-6f3b-48a5-9cbf-38109953b55a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:09.827255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b331550-6f3b-48a5-9cbf-38109953b55a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:09.827635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 391/799] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:31:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:09.827635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d04c4-c12f-4980-99d5-bdc18807d937 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:09.842227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b331550-6f3b-48a5-9cbf-38109953b55a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:09.843647 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b331550-6f3b-48a5-9cbf-38109953b55a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-7b331550-6f3b-48a5-9cbf-38109953b55a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:09.861658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:09.863553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:09.865362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:09.865362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:09.865362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:09.865362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 409/800] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:09.865362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b331550-6f3b-48a5-9cbf-38109953b55a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:10.008136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:10.013763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d04c4-c12f-4980-99d5-bdc18807d937 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:10.014315 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b331550-6f3b-48a5-9cbf-38109953b55a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:10.066839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a423b-eb6b-4185-aa1b-5dbc0cf4d1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:10.067783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a423b-eb6b-4185-aa1b-5dbc0cf4d1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:10.068185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a423b-eb6b-4185-aa1b-5dbc0cf4d1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:10.068595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a423b-eb6b-4185-aa1b-5dbc0cf4d1d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:10.077011 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a489c57-5663-46d8-a442-edfe52a39da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:10.079319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a489c57-5663-46d8-a442-edfe52a39da3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:10.079814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a489c57-5663-46d8-a442-edfe52a39da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:10.080300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a489c57-5663-46d8-a442-edfe52a39da3 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:10.081424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a489c57-5663-46d8-a442-edfe52a39da3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:10.081960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a489c57-5663-46d8-a442-edfe52a39da3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:10.083752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a423b-eb6b-4185-aa1b-5dbc0cf4d1d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:10.084591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a489c57-5663-46d8-a442-edfe52a39da3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:10.085500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a423b-eb6b-4185-aa1b-5dbc0cf4d1d2 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:10.099823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 410/801] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Aug 29 23:31:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:10.099823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a489c57-5663-46d8-a442-edfe52a39da3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-d01a423b-eb6b-4185-aa1b-5dbc0cf4d1d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:10.109372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:10.111548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:10.113343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:10.113343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:10.113343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:10.117654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 392/802] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:10.118966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a423b-eb6b-4185-aa1b-5dbc0cf4d1d2 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:10.153535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:10.273364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a423b-eb6b-4185-aa1b-5dbc0cf4d1d2 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:10.282527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a489c57-5663-46d8-a442-edfe52a39da3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:10.329019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3860d1d-b85c-4911-bde7-29c488391680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:10.329753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3860d1d-b85c-4911-bde7-29c488391680 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:10.330298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3860d1d-b85c-4911-bde7-29c488391680 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:10.330525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3860d1d-b85c-4911-bde7-29c488391680 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:10.331243 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3860d1d-b85c-4911-bde7-29c488391680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:10.331619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3860d1d-b85c-4911-bde7-29c488391680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:10.333110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3860d1d-b85c-4911-bde7-29c488391680 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:10.348114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 393/803] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:10.349128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3860d1d-b85c-4911-bde7-29c488391680 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:10.364934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcc7d5-4e64-4dcd-9ebc-3525dec122a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:10.366856 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcc7d5-4e64-4dcd-9ebc-3525dec122a6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:10.367675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcc7d5-4e64-4dcd-9ebc-3525dec122a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:10.368523 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcc7d5-4e64-4dcd-9ebc-3525dec122a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:10.369766 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcc7d5-4e64-4dcd-9ebc-3525dec122a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:10.370279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcc7d5-4e64-4dcd-9ebc-3525dec122a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:10.372938 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcc7d5-4e64-4dcd-9ebc-3525dec122a6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:10.390553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 411/804] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:31:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:10.390553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcc7d5-4e64-4dcd-9ebc-3525dec122a6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:10.439391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:10.485319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:10.506446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3860d1d-b85c-4911-bde7-29c488391680 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:10.537013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcc7d5-4e64-4dcd-9ebc-3525dec122a6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:10.551660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfeade-45fd-4d90-9225-7834a857c0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:10.552272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfeade-45fd-4d90-9225-7834a857c0fe None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:10.552554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfeade-45fd-4d90-9225-7834a857c0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:10.552830 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfeade-45fd-4d90-9225-7834a857c0fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:10.553495 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfeade-45fd-4d90-9225-7834a857c0fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:10.554938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfeade-45fd-4d90-9225-7834a857c0fe None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:10.584875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb980b-a5f2-47a6-bad9-acff6f43da1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:10.585927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb980b-a5f2-47a6-bad9-acff6f43da1e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:10.586448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb980b-a5f2-47a6-bad9-acff6f43da1e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:10.586974 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb980b-a5f2-47a6-bad9-acff6f43da1e None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:10.588170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb980b-a5f2-47a6-bad9-acff6f43da1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:10.588728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb980b-a5f2-47a6-bad9-acff6f43da1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:10.591342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb980b-a5f2-47a6-bad9-acff6f43da1e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:10.618120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 412/805] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:10.618120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb980b-a5f2-47a6-bad9-acff6f43da1e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:10.654264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 394/806] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:31:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:31:10.655148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfeade-45fd-4d90-9225-7834a857c0fe None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:10.758742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:10.779277 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb980b-a5f2-47a6-bad9-acff6f43da1e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:10.813850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfeade-45fd-4d90-9225-7834a857c0fe None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:10.832558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301728f-3ad4-404d-8625-e74ea3fde9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:10.833216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301728f-3ad4-404d-8625-e74ea3fde9ec None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:10.833495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301728f-3ad4-404d-8625-e74ea3fde9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:10.833775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301728f-3ad4-404d-8625-e74ea3fde9ec None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:10.842295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301728f-3ad4-404d-8625-e74ea3fde9ec None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:10.843897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301728f-3ad4-404d-8625-e74ea3fde9ec None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-9301728f-3ad4-404d-8625-e74ea3fde9ec None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:10.860916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:10.863022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:10.864839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:10.864839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:10.864839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:10.864839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:10.864839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:10.864839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:10.864839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:10.864839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:10.864839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:31:10.864839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:10.864839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 413/807] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Aug 29 23:31:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:10.866011 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e251b94-6707-4ba2-901d-75b3a3005522 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:10.866011 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e251b94-6707-4ba2-901d-75b3a3005522 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:10.866258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e251b94-6707-4ba2-901d-75b3a3005522 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:10.868529 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 414/808] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:10.869881 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e251b94-6707-4ba2-901d-75b3a3005522 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:10.877459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809958e-8c41-404d-a7fb-f646f45c1b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:10.878171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809958e-8c41-404d-a7fb-f646f45c1b4e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:10.878586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809958e-8c41-404d-a7fb-f646f45c1b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:10.878976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809958e-8c41-404d-a7fb-f646f45c1b4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:10.882722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:10.898638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809958e-8c41-404d-a7fb-f646f45c1b4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:10.900933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809958e-8c41-404d-a7fb-f646f45c1b4e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-1809958e-8c41-404d-a7fb-f646f45c1b4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:10.928701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:10.932274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:10.933910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:10.933910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:10.933910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:10.933910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 395/809] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:10.933910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809958e-8c41-404d-a7fb-f646f45c1b4e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:11.065102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e251b94-6707-4ba2-901d-75b3a3005522 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:11.072352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809958e-8c41-404d-a7fb-f646f45c1b4e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:11.115422 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd68142-d5e7-4807-b45e-4b3844c17a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:11.116242 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd68142-d5e7-4807-b45e-4b3844c17a1e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:11.116649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd68142-d5e7-4807-b45e-4b3844c17a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:11.117155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd68142-d5e7-4807-b45e-4b3844c17a1e None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:11.118189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd68142-d5e7-4807-b45e-4b3844c17a1e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:11.118661 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd68142-d5e7-4807-b45e-4b3844c17a1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:11.120440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd68142-d5e7-4807-b45e-4b3844c17a1e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:11.134120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a234f79-a935-44fe-9be8-dc1ff2755b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:11.135252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a234f79-a935-44fe-9be8-dc1ff2755b0e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:11.135730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a234f79-a935-44fe-9be8-dc1ff2755b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:11.136198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a234f79-a935-44fe-9be8-dc1ff2755b0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:11.151032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 396/810] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Aug 29 23:31:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:11.151032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a4a1b-b322-41e0-92f4-85cdff2eec8f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:11.151032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a4a1b-b322-41e0-92f4-85cdff2eec8f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:11.151032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a4a1b-b322-41e0-92f4-85cdff2eec8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:11.153873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a234f79-a935-44fe-9be8-dc1ff2755b0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:11.157024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a234f79-a935-44fe-9be8-dc1ff2755b0e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:11.173823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-755a4a1b-b322-41e0-92f4-85cdff2eec8f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:11.175313 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-755a4a1b-b322-41e0-92f4-85cdff2eec8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-6a234f79-a935-44fe-9be8-dc1ff2755b0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:11.179581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:11.181368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:11.183991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:11.183991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:11.183991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:11.183991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:11.183991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:11.183991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:11.183991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:11.183991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:11.190337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 415/811] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:11.191742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a234f79-a935-44fe-9be8-dc1ff2755b0e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:11.214913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:11.238636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:11.272147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 397/812] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:11.273967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a4a1b-b322-41e0-92f4-85cdff2eec8f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:11.481296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a4a1b-b322-41e0-92f4-85cdff2eec8f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:11.557561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089791f4-6b4c-4488-952e-0eb8d18c1439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:11.558228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089791f4-6b4c-4488-952e-0eb8d18c1439 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:11.558532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089791f4-6b4c-4488-952e-0eb8d18c1439 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:11.558804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089791f4-6b4c-4488-952e-0eb8d18c1439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:11.571409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089791f4-6b4c-4488-952e-0eb8d18c1439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:11.572916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089791f4-6b4c-4488-952e-0eb8d18c1439 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:11.602293 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a234f79-a935-44fe-9be8-dc1ff2755b0e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-089791f4-6b4c-4488-952e-0eb8d18c1439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:11.611025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:11.612963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:11.614969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:11.614969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:11.614969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:11.614969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 398/813] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:11.616906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089791f4-6b4c-4488-952e-0eb8d18c1439 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:11.650189 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d1d45-f837-4640-ba92-4341895ed5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:11.650976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d1d45-f837-4640-ba92-4341895ed5ff None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:11.651522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d1d45-f837-4640-ba92-4341895ed5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:11.651900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d1d45-f837-4640-ba92-4341895ed5ff None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:11.652696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d1d45-f837-4640-ba92-4341895ed5ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:11.653526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d1d45-f837-4640-ba92-4341895ed5ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:11.655292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d1d45-f837-4640-ba92-4341895ed5ff None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:11.671537 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 416/814] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:11.672835 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d1d45-f837-4640-ba92-4341895ed5ff None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:11.692927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:11.766642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:11.806956 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d1d45-f837-4640-ba92-4341895ed5ff None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:11.855259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e73451-3067-48c0-b6d7-31e348b58ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:11.856089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e73451-3067-48c0-b6d7-31e348b58ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:11.856518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e73451-3067-48c0-b6d7-31e348b58ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:11.856921 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e73451-3067-48c0-b6d7-31e348b58ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:11.857755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e73451-3067-48c0-b6d7-31e348b58ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:11.858132 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089791f4-6b4c-4488-952e-0eb8d18c1439 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:11.858303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e73451-3067-48c0-b6d7-31e348b58ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:11.859968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e73451-3067-48c0-b6d7-31e348b58ba4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:11.875741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 417/815] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:11.878531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e73451-3067-48c0-b6d7-31e348b58ba4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:11.932600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2d529-0744-4034-8e94-f52f75349596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:11.933847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d529-0744-4034-8e94-f52f75349596 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:11.934361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d529-0744-4034-8e94-f52f75349596 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:11.935004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d529-0744-4034-8e94-f52f75349596 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:11.936387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2d529-0744-4034-8e94-f52f75349596 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:11.937575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2d529-0744-4034-8e94-f52f75349596 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:11.940777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2d529-0744-4034-8e94-f52f75349596 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:11.970702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 399/816] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:11.970702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2d529-0744-4034-8e94-f52f75349596 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:12.074696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e73451-3067-48c0-b6d7-31e348b58ba4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:12.104872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:12.113862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:12.122738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2d529-0744-4034-8e94-f52f75349596 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:12.130866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978fb71-b8a3-4bb7-910c-1ae1825f9b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:12.131824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978fb71-b8a3-4bb7-910c-1ae1825f9b77 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:12.132322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978fb71-b8a3-4bb7-910c-1ae1825f9b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:12.133390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978fb71-b8a3-4bb7-910c-1ae1825f9b77 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:12.147579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978fb71-b8a3-4bb7-910c-1ae1825f9b77 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:12.151550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978fb71-b8a3-4bb7-910c-1ae1825f9b77 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:12.169301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624bfe8-f5a1-4a7e-9324-c31e9e8bb6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:12.170152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624bfe8-f5a1-4a7e-9324-c31e9e8bb6dc None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:12.170629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624bfe8-f5a1-4a7e-9324-c31e9e8bb6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:12.171648 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624bfe8-f5a1-4a7e-9324-c31e9e8bb6dc None admin] PATH_INFO: `/v3/services/network` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:12.172460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624bfe8-f5a1-4a7e-9324-c31e9e8bb6dc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:12.174677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624bfe8-f5a1-4a7e-9324-c31e9e8bb6dc None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c978fb71-b8a3-4bb7-910c-1ae1825f9b77 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:12.175192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:12.177093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:12.178626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:12.178626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:12.178626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:12.178626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:12.178626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:12.178626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:12.178626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:12.178626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:12.178626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:31:12.178626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:12.187871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 418/817] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Aug 29 23:31:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:12.187871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978fb71-b8a3-4bb7-910c-1ae1825f9b77 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-1624bfe8-f5a1-4a7e-9324-c31e9e8bb6dc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:12.194927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:12.196889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 23:31:12.199436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:12.205177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 400/818] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:11 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:12.206228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624bfe8-f5a1-4a7e-9324-c31e9e8bb6dc None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:12.372493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624bfe8-f5a1-4a7e-9324-c31e9e8bb6dc None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:12.376247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978fb71-b8a3-4bb7-910c-1ae1825f9b77 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:12.432827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b8944-ba78-47f1-9f4d-df85b5df3f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:12.433520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b8944-ba78-47f1-9f4d-df85b5df3f7e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:12.433855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b55a9-ccbe-44b6-b7f7-ee305825f6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:12.434042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b8944-ba78-47f1-9f4d-df85b5df3f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:12.434321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b8944-ba78-47f1-9f4d-df85b5df3f7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:12.435502 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b55a9-ccbe-44b6-b7f7-ee305825f6a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:12.436013 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b55a9-ccbe-44b6-b7f7-ee305825f6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:12.436560 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b55a9-ccbe-44b6-b7f7-ee305825f6a4 None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/c6b37b09a1ae45e8bc16adfa59500eb9/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:12.440541 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b55a9-ccbe-44b6-b7f7-ee305825f6a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=c6b37b09a1ae45e8bc16adfa59500eb9, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:12.443329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b55a9-ccbe-44b6-b7f7-ee305825f6a4 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:12.443528 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b8944-ba78-47f1-9f4d-df85b5df3f7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:12.445219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b8944-ba78-47f1-9f4d-df85b5df3f7e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-520b8944-ba78-47f1-9f4d-df85b5df3f7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:12.459282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:12.473050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:12.474787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:12.474787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:12.474787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:12.474787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:12.474787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:12.474787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:12.474787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:12.474787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:12.474787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 419/819] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:12.474787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b8944-ba78-47f1-9f4d-df85b5df3f7e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:12.527838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:12.590120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 401/820] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:31:12 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/c6b37b09a1ae45e8bc16adfa59500eb9/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:31:12.592079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b55a9-ccbe-44b6-b7f7-ee305825f6a4 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:12.600062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b8944-ba78-47f1-9f4d-df85b5df3f7e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:12.674873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:12.681241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab65ce-a992-4f67-97c1-2e98d8b9af7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:12.682522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab65ce-a992-4f67-97c1-2e98d8b9af7a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:12.683574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab65ce-a992-4f67-97c1-2e98d8b9af7a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:12.684092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab65ce-a992-4f67-97c1-2e98d8b9af7a None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:12.685708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab65ce-a992-4f67-97c1-2e98d8b9af7a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:12.686341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab65ce-a992-4f67-97c1-2e98d8b9af7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:12.690557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab65ce-a992-4f67-97c1-2e98d8b9af7a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:12.735434 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 420/821] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Aug 29 23:31:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:12.737218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab65ce-a992-4f67-97c1-2e98d8b9af7a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:12.819276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b55a9-ccbe-44b6-b7f7-ee305825f6a4 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:12.861569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04784f71-9261-451a-9523-ebd31f4aab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:12.862175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04784f71-9261-451a-9523-ebd31f4aab70 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:12.862500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04784f71-9261-451a-9523-ebd31f4aab70 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:12.862784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04784f71-9261-451a-9523-ebd31f4aab70 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:12.863438 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04784f71-9261-451a-9523-ebd31f4aab70 None admin] RBAC: Adding query filter params (name=network) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:12.863745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04784f71-9261-451a-9523-ebd31f4aab70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:12.865140 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04784f71-9261-451a-9523-ebd31f4aab70 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:12.870981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 402/822] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:31:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:12.871925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04784f71-9261-451a-9523-ebd31f4aab70 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:12.907350 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab65ce-a992-4f67-97c1-2e98d8b9af7a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:12.911262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:12.979437 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:12.993567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a75ed9-069d-4d07-9c20-f47a426a34d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:12.994494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a75ed9-069d-4d07-9c20-f47a426a34d6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:12.994915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a75ed9-069d-4d07-9c20-f47a426a34d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:12.995322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a75ed9-069d-4d07-9c20-f47a426a34d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:12.996404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a75ed9-069d-4d07-9c20-f47a426a34d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:12.996868 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a75ed9-069d-4d07-9c20-f47a426a34d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:12.999335 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a75ed9-069d-4d07-9c20-f47a426a34d6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:13.012316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04784f71-9261-451a-9523-ebd31f4aab70 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:13.023192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 421/823] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:13.023192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a75ed9-069d-4d07-9c20-f47a426a34d6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:13.065214 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fcc9e-a613-4084-93c6-5e0fdfc16a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:13.065949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fcc9e-a613-4084-93c6-5e0fdfc16a40 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:13.066297 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fcc9e-a613-4084-93c6-5e0fdfc16a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:13.066663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fcc9e-a613-4084-93c6-5e0fdfc16a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:13.149576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a75ed9-069d-4d07-9c20-f47a426a34d6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:13.158613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fcc9e-a613-4084-93c6-5e0fdfc16a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:13.160881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fcc9e-a613-4084-93c6-5e0fdfc16a40 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:13.200554 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e94a70-a5e0-41d0-80f6-afbb178875b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:13.201597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e94a70-a5e0-41d0-80f6-afbb178875b2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:13.202037 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e94a70-a5e0-41d0-80f6-afbb178875b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:13.203771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e94a70-a5e0-41d0-80f6-afbb178875b2 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:13.204958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e94a70-a5e0-41d0-80f6-afbb178875b2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:13.205409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e94a70-a5e0-41d0-80f6-afbb178875b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:13.207967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e94a70-a5e0-41d0-80f6-afbb178875b2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:13.222476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 422/824] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:31:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:13.222476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84a8a3-4784-439b-a888-2d124e530703 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:13.222476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84a8a3-4784-439b-a888-2d124e530703 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:13.222476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84a8a3-4784-439b-a888-2d124e530703 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:13.223850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 423/825] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:13.225280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84a8a3-4784-439b-a888-2d124e530703 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:13.269858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:13.317204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:13.355517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 403/826] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:31:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:31:13.356704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fcc9e-a613-4084-93c6-5e0fdfc16a40 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:13.412271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84a8a3-4784-439b-a888-2d124e530703 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:13.466090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ed136-db6b-4817-8d83-5b3ab6a36469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:13.466817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ed136-db6b-4817-8d83-5b3ab6a36469 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:13.467203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ed136-db6b-4817-8d83-5b3ab6a36469 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:13.467510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ed136-db6b-4817-8d83-5b3ab6a36469 None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/ff5b4f2f41604a8286141bce23db35c5/roles/f0926be4d3d4462fac854212e101ccdd` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:13.487698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ed136-db6b-4817-8d83-5b3ab6a36469 None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=ff5b4f2f41604a8286141bce23db35c5, role_id=f0926be4d3d4462fac854212e101ccdd)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:13.489232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ed136-db6b-4817-8d83-5b3ab6a36469 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:13.509479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fcc9e-a613-4084-93c6-5e0fdfc16a40 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:13.548035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea43f92-5dd0-4a02-b80f-85584b9b5596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:13.548571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea43f92-5dd0-4a02-b80f-85584b9b5596 None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:13.548848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea43f92-5dd0-4a02-b80f-85584b9b5596 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:13.549118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea43f92-5dd0-4a02-b80f-85584b9b5596 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:13.549844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea43f92-5dd0-4a02-b80f-85584b9b5596 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:13.551312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea43f92-5dd0-4a02-b80f-85584b9b5596 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:13.570466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 424/827] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:31:13 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/ff5b4f2f41604a8286141bce23db35c5/roles/f0926be4d3d4462fac854212e101ccdd => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:31:13.570466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113760ea-0147-4bea-8bf8-aecb01658299 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:13.570466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113760ea-0147-4bea-8bf8-aecb01658299 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:13.570466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113760ea-0147-4bea-8bf8-aecb01658299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:13.593662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-113760ea-0147-4bea-8bf8-aecb01658299 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:13.595737 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-113760ea-0147-4bea-8bf8-aecb01658299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:13.648582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 404/828] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:31:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:31:13.651153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea43f92-5dd0-4a02-b80f-85584b9b5596 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:13.735732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:13.814039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 425/829] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:13.895353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea43f92-5dd0-4a02-b80f-85584b9b5596 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:13.937068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd1367-729e-4754-a1fb-b0e10ce212b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:13.937721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1367-729e-4754-a1fb-b0e10ce212b5 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:13.938000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1367-729e-4754-a1fb-b0e10ce212b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:13.938270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1367-729e-4754-a1fb-b0e10ce212b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:13.951435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd1367-729e-4754-a1fb-b0e10ce212b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:13.953134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd1367-729e-4754-a1fb-b0e10ce212b5 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-5bdd1367-729e-4754-a1fb-b0e10ce212b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:13.970283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:13.971373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:13.972273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:13.972273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:13.972273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:13.972273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 405/830] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:13.980451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113760ea-0147-4bea-8bf8-aecb01658299 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:13.984813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:14.111388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113760ea-0147-4bea-8bf8-aecb01658299 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:14.152679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c839d81-4ac5-4900-a3ca-66aaff02b43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:14.153293 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c839d81-4ac5-4900-a3ca-66aaff02b43d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:14.153566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c839d81-4ac5-4900-a3ca-66aaff02b43d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:14.153833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c839d81-4ac5-4900-a3ca-66aaff02b43d None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:14.154504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c839d81-4ac5-4900-a3ca-66aaff02b43d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:14.154808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c839d81-4ac5-4900-a3ca-66aaff02b43d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:14.156792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c839d81-4ac5-4900-a3ca-66aaff02b43d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:14.170572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 426/831] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:14.179246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd1367-729e-4754-a1fb-b0e10ce212b5 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:14.299358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:14.321722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd1367-729e-4754-a1fb-b0e10ce212b5 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:14.343885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c839d81-4ac5-4900-a3ca-66aaff02b43d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:14.370133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1209c45-e24d-45df-ace4-3af3ce02a4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:14.371315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1209c45-e24d-45df-ace4-3af3ce02a4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:14.371595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1209c45-e24d-45df-ace4-3af3ce02a4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:14.371926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1209c45-e24d-45df-ace4-3af3ce02a4c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:14.382130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1209c45-e24d-45df-ace4-3af3ce02a4c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:14.383987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1209c45-e24d-45df-ace4-3af3ce02a4c6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-c1209c45-e24d-45df-ace4-3af3ce02a4c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:14.401446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:14.402412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:14.403354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:14.403354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:14.403354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:14.403354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:14.403354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:14.403354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:14.403354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:14.403354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:14.407667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 406/832] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:14.412869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1209c45-e24d-45df-ace4-3af3ce02a4c6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:14.450479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:14.533660 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c839d81-4ac5-4900-a3ca-66aaff02b43d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:14.603981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1209c45-e24d-45df-ace4-3af3ce02a4c6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:14.611287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9448e9-5812-4fdc-966a-33525d54fccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:14.612631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9448e9-5812-4fdc-966a-33525d54fccc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:14.613539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9448e9-5812-4fdc-966a-33525d54fccc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:14.614258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9448e9-5812-4fdc-966a-33525d54fccc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:14.615589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9448e9-5812-4fdc-966a-33525d54fccc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:14.618547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9448e9-5812-4fdc-966a-33525d54fccc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-6e9448e9-5812-4fdc-966a-33525d54fccc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:14.637930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:14.641303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 23:31:14.643932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:14.654041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 427/833] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:31:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:14.659930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9448e9-5812-4fdc-966a-33525d54fccc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:14.677458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225633-3d25-4a1e-9eea-fca2a264989e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:14.678634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225633-3d25-4a1e-9eea-fca2a264989e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:14.679308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225633-3d25-4a1e-9eea-fca2a264989e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:14.680793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225633-3d25-4a1e-9eea-fca2a264989e None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:14.682208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28225633-3d25-4a1e-9eea-fca2a264989e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:14.682879 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28225633-3d25-4a1e-9eea-fca2a264989e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:14.687450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28225633-3d25-4a1e-9eea-fca2a264989e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:14.722467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 407/834] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:14.726983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225633-3d25-4a1e-9eea-fca2a264989e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:14.734737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:14.871533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9448e9-5812-4fdc-966a-33525d54fccc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:14.905608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:14.914166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43209a5-ae55-4f80-85c9-f7d751d94acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:14.914857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43209a5-ae55-4f80-85c9-f7d751d94acc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:14.915672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43209a5-ae55-4f80-85c9-f7d751d94acc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:14.915672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43209a5-ae55-4f80-85c9-f7d751d94acc None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:14.916065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43209a5-ae55-4f80-85c9-f7d751d94acc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:14.916372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43209a5-ae55-4f80-85c9-f7d751d94acc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:14.917852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43209a5-ae55-4f80-85c9-f7d751d94acc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:14.929647 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 428/835] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:31:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:14.931460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43209a5-ae55-4f80-85c9-f7d751d94acc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:14.976476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225633-3d25-4a1e-9eea-fca2a264989e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:15.043679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43209a5-ae55-4f80-85c9-f7d751d94acc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:15.051868 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d14f7f-374c-47b0-8fb8-779d27ea337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:15.052898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d14f7f-374c-47b0-8fb8-779d27ea337f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:15.053384 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d14f7f-374c-47b0-8fb8-779d27ea337f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:15.053862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d14f7f-374c-47b0-8fb8-779d27ea337f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:15.055259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d14f7f-374c-47b0-8fb8-779d27ea337f None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=e4a8261d92064b838fc6e8f415879566) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:15.055797 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d14f7f-374c-47b0-8fb8-779d27ea337f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:15.058520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d14f7f-374c-47b0-8fb8-779d27ea337f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:15.070192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 408/836] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:31:14 2022] GET /identity/v3/role_assignments?user.id=e4a8261d92064b838fc6e8f415879566&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=d86dba8efe20423abdde06031690beb8 => generated 608 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:15.074497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f0977-ffd8-4b89-bf7a-aa79db475611 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:15.074497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f0977-ffd8-4b89-bf7a-aa79db475611 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:15.074497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f0977-ffd8-4b89-bf7a-aa79db475611 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:15.078480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 409/837] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:15.083407 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a21c8-f75b-4d04-9dd3-5bfabfa0aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:15.084376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a21c8-f75b-4d04-9dd3-5bfabfa0aa99 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:15.084855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4204d9-c564-4180-95f2-a5748aca2683 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:15.084971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a21c8-f75b-4d04-9dd3-5bfabfa0aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:15.085145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4204d9-c564-4180-95f2-a5748aca2683 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:15.085545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4204d9-c564-4180-95f2-a5748aca2683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:15.085778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a21c8-f75b-4d04-9dd3-5bfabfa0aa99 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:15.086679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a21c8-f75b-4d04-9dd3-5bfabfa0aa99 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:15.087179 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a21c8-f75b-4d04-9dd3-5bfabfa0aa99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:15.088832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a21c8-f75b-4d04-9dd3-5bfabfa0aa99 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:15.095172 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 429/838] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:31:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:15.106772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-ff4204d9-c564-4180-95f2-a5748aca2683 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:15.108694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ff4204d9-c564-4180-95f2-a5748aca2683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:15.194025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:15.200144 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 410/839] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:15.476657 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a21c8-f75b-4d04-9dd3-5bfabfa0aa99 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:15.579412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:15.660950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a21c8-f75b-4d04-9dd3-5bfabfa0aa99 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:15.727234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eba28-e0b0-4640-a86a-b096eb535593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:15.728230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eba28-e0b0-4640-a86a-b096eb535593 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:15.728723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eba28-e0b0-4640-a86a-b096eb535593 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:15.729213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eba28-e0b0-4640-a86a-b096eb535593 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:15.748174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1eba28-e0b0-4640-a86a-b096eb535593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-cf1eba28-e0b0-4640-a86a-b096eb535593 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:15.758145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:15.759194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:31:15.760033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:15.761229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 430/840] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:31:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:15.772100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4204d9-c564-4180-95f2-a5748aca2683 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:15.982753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4204d9-c564-4180-95f2-a5748aca2683 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:16.054266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e27ab-5561-42af-9292-800de80d4c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:16.055288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e27ab-5561-42af-9292-800de80d4c0f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:16.055771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e27ab-5561-42af-9292-800de80d4c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:16.056288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e27ab-5561-42af-9292-800de80d4c0f None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:16.057497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e27ab-5561-42af-9292-800de80d4c0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:16.058035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e27ab-5561-42af-9292-800de80d4c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:16.060607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e27ab-5561-42af-9292-800de80d4c0f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:16.078266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 411/841] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:31:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:16.087613 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eba28-e0b0-4640-a86a-b096eb535593 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:16.278510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eba28-e0b0-4640-a86a-b096eb535593 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:16.292797 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249282a-c788-4a49-8c09-fee5453be56e None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:16.293253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249282a-c788-4a49-8c09-fee5453be56e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:16.293729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249282a-c788-4a49-8c09-fee5453be56e None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:16.303471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 412/842] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:31:16.306351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f03f7-4504-44b2-8369-5375f5487162 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:16.306846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f03f7-4504-44b2-8369-5375f5487162 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:16.307323 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f03f7-4504-44b2-8369-5375f5487162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:16.311951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:16.324805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4e01b-1cc8-45aa-8e28-324be724c739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:16.325460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4e01b-1cc8-45aa-8e28-324be724c739 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:16.325735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4e01b-1cc8-45aa-8e28-324be724c739 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:16.326008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4e01b-1cc8-45aa-8e28-324be724c739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:16.340247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4e01b-1cc8-45aa-8e28-324be724c739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:16.341725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4e01b-1cc8-45aa-8e28-324be724c739 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:16.359876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-130f03f7-4504-44b2-8369-5375f5487162 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:16.361105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-130f03f7-4504-44b2-8369-5375f5487162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-afb4e01b-1cc8-45aa-8e28-324be724c739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:16.378973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:16.380788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:16.383219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:16.383219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:16.383219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:16.384765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 431/843] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:16.395423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4e01b-1cc8-45aa-8e28-324be724c739 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:16.481226 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:16.505983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 413/844] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:16.616533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4e01b-1cc8-45aa-8e28-324be724c739 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:16.685048 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9bc0-698a-4688-9c02-8376e613a453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:16.686075 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9bc0-698a-4688-9c02-8376e613a453 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:16.687436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9bc0-698a-4688-9c02-8376e613a453 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:16.687436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9bc0-698a-4688-9c02-8376e613a453 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:16.690011 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d9bc0-698a-4688-9c02-8376e613a453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:16.690597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d9bc0-698a-4688-9c02-8376e613a453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:16.693416 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d9bc0-698a-4688-9c02-8376e613a453 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:16.716162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 432/845] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:16.722932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f03f7-4504-44b2-8369-5375f5487162 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:16.853026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9bc0-698a-4688-9c02-8376e613a453 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:16.867034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f03f7-4504-44b2-8369-5375f5487162 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:16.912044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc04251-fd9c-455c-91db-593a649b429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:16.912982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc04251-fd9c-455c-91db-593a649b429a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:16.913482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc04251-fd9c-455c-91db-593a649b429a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:16.913873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc04251-fd9c-455c-91db-593a649b429a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:16.923791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc04251-fd9c-455c-91db-593a649b429a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:16.924541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:16.925282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc04251-fd9c-455c-91db-593a649b429a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-adc04251-fd9c-455c-91db-593a649b429a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:16.942520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:16.955472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 414/846] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:31:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:16.957295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc04251-fd9c-455c-91db-593a649b429a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:17.048368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9bc0-698a-4688-9c02-8376e613a453 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:17.079632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:17.148476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25161fc-bb38-4e38-92b7-d08e1da89236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:17.150903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25161fc-bb38-4e38-92b7-d08e1da89236 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:17.151911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25161fc-bb38-4e38-92b7-d08e1da89236 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:17.153223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25161fc-bb38-4e38-92b7-d08e1da89236 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:17.171749 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc04251-fd9c-455c-91db-593a649b429a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:17.179979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25161fc-bb38-4e38-92b7-d08e1da89236 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d25161fc-bb38-4e38-92b7-d08e1da89236 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:17.195562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:17.197389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:31:17.199099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:17.203643 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 433/847] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:31:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:17.211638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25161fc-bb38-4e38-92b7-d08e1da89236 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:17.227057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e6ae1-3ac9-4fae-a7a7-1b9b2ba8462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:17.227671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e6ae1-3ac9-4fae-a7a7-1b9b2ba8462d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:17.227949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e6ae1-3ac9-4fae-a7a7-1b9b2ba8462d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:17.228220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e6ae1-3ac9-4fae-a7a7-1b9b2ba8462d None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:17.228959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e6ae1-3ac9-4fae-a7a7-1b9b2ba8462d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:17.229259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e6ae1-3ac9-4fae-a7a7-1b9b2ba8462d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:17.230953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e6ae1-3ac9-4fae-a7a7-1b9b2ba8462d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:17.258456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 415/848] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Aug 29 23:31:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:17.262086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e6ae1-3ac9-4fae-a7a7-1b9b2ba8462d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:17.397303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:17.420390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25161fc-bb38-4e38-92b7-d08e1da89236 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:17.455605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e6ae1-3ac9-4fae-a7a7-1b9b2ba8462d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:17.476042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:17.492056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18655a80-663c-49e7-9991-b43e9b649f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:17.494231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18655a80-663c-49e7-9991-b43e9b649f26 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:17.495055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18655a80-663c-49e7-9991-b43e9b649f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:17.495790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18655a80-663c-49e7-9991-b43e9b649f26 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:17.497278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18655a80-663c-49e7-9991-b43e9b649f26 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:17.498085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18655a80-663c-49e7-9991-b43e9b649f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:17.502809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18655a80-663c-49e7-9991-b43e9b649f26 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:17.525119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c686cb-86c5-469a-a204-5465b2756587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:17.526139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c686cb-86c5-469a-a204-5465b2756587 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:17.526765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c686cb-86c5-469a-a204-5465b2756587 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:17.527326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c686cb-86c5-469a-a204-5465b2756587 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:17.534730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 434/849] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:31:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:17.541854 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18655a80-663c-49e7-9991-b43e9b649f26 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:17.674838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c686cb-86c5-469a-a204-5465b2756587 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:17.677575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c686cb-86c5-469a-a204-5465b2756587 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:17.689780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18655a80-663c-49e7-9991-b43e9b649f26 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:17.739473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92241a9-b7d7-456b-9339-5e3188d13dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:17.740978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92241a9-b7d7-456b-9339-5e3188d13dcf None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:17.741470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92241a9-b7d7-456b-9339-5e3188d13dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:17.741951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92241a9-b7d7-456b-9339-5e3188d13dcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:17.762800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92241a9-b7d7-456b-9339-5e3188d13dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:17.767045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92241a9-b7d7-456b-9339-5e3188d13dcf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e92241a9-b7d7-456b-9339-5e3188d13dcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:17.785503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:17.795397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:17.797608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:17.797608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:17.797608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:17.797608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 435/850] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:17.797608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa262192-7ec7-475f-90ea-bbaa3fa0a7a9 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:17.797608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa262192-7ec7-475f-90ea-bbaa3fa0a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:17.797608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa262192-7ec7-475f-90ea-bbaa3fa0a7a9 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:17.797608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 436/851] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:17.797608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa262192-7ec7-475f-90ea-bbaa3fa0a7a9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:17.799873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:17.898471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 416/852] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:31:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:31:17.898471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacce48b-339e-4de3-b984-c11ef84df5de None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:17.898471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacce48b-339e-4de3-b984-c11ef84df5de None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:17.898471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacce48b-339e-4de3-b984-c11ef84df5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:17.921038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-cacce48b-339e-4de3-b984-c11ef84df5de None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:17.922260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-cacce48b-339e-4de3-b984-c11ef84df5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:17.924669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:17.949906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa262192-7ec7-475f-90ea-bbaa3fa0a7a9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:18.015527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0f008-866f-4103-9164-7ad596b7984a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:18.016601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0f008-866f-4103-9164-7ad596b7984a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:18.017087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0f008-866f-4103-9164-7ad596b7984a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:18.017535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0f008-866f-4103-9164-7ad596b7984a None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:18.018714 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0f008-866f-4103-9164-7ad596b7984a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:18.019258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0f008-866f-4103-9164-7ad596b7984a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:18.021882 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0f008-866f-4103-9164-7ad596b7984a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:18.046463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 437/853] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:18.046463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0f008-866f-4103-9164-7ad596b7984a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:18.066869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 417/854] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:18.068314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacce48b-339e-4de3-b984-c11ef84df5de None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:18.122770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:18.200936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0f008-866f-4103-9164-7ad596b7984a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:18.221607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacce48b-339e-4de3-b984-c11ef84df5de None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:18.271933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d541dc-9255-4cc3-9f20-a979e0c6e8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:18.272610 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d541dc-9255-4cc3-9f20-a979e0c6e8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:18.272890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d541dc-9255-4cc3-9f20-a979e0c6e8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:18.273158 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d541dc-9255-4cc3-9f20-a979e0c6e8c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:18.283233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d541dc-9255-4cc3-9f20-a979e0c6e8c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:18.284694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d541dc-9255-4cc3-9f20-a979e0c6e8c8 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:18.287256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:18.293331 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83768511-231d-421b-8b40-816dbb631248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:18.295869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83768511-231d-421b-8b40-816dbb631248 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:18.296706 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83768511-231d-421b-8b40-816dbb631248 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:18.302085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83768511-231d-421b-8b40-816dbb631248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-33d541dc-9255-4cc3-9f20-a979e0c6e8c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:18.306743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:18.308325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:18.312374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:18.312374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:18.312374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:18.312374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:18.312374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:18.312374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:18.312374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:18.312374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:18.312374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:31:18.312374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:18.312374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 418/855] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Aug 29 23:31:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:18.321541 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d541dc-9255-4cc3-9f20-a979e0c6e8c8 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:18.330815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83768511-231d-421b-8b40-816dbb631248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:18.332899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83768511-231d-421b-8b40-816dbb631248 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-83768511-231d-421b-8b40-816dbb631248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:18.360576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:18.361702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:18.362624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:18.362624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:18.362624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:18.364885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 438/856] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:18.373795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83768511-231d-421b-8b40-816dbb631248 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:18.513114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83768511-231d-421b-8b40-816dbb631248 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:18.531092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d541dc-9255-4cc3-9f20-a979e0c6e8c8 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:18.533409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:18.578938 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095720a8-bb0b-4dc5-b38e-362d31908a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:18.579557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095720a8-bb0b-4dc5-b38e-362d31908a49 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:18.579838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095720a8-bb0b-4dc5-b38e-362d31908a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:18.580127 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095720a8-bb0b-4dc5-b38e-362d31908a49 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:18.580770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095720a8-bb0b-4dc5-b38e-362d31908a49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:18.581078 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095720a8-bb0b-4dc5-b38e-362d31908a49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:18.582506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095720a8-bb0b-4dc5-b38e-362d31908a49 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:18.588819 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8484d-b9c7-4e13-a12a-0ebbcea80f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:18.589845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8484d-b9c7-4e13-a12a-0ebbcea80f75 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:18.590335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8484d-b9c7-4e13-a12a-0ebbcea80f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:18.590859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8484d-b9c7-4e13-a12a-0ebbcea80f75 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:18.593272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8484d-b9c7-4e13-a12a-0ebbcea80f75 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:18.594180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8484d-b9c7-4e13-a12a-0ebbcea80f75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:18.596688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8484d-b9c7-4e13-a12a-0ebbcea80f75 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:18.596881 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 439/857] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:18.597975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334710-4ea9-428c-9153-07f28bd9d48b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:18.598257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334710-4ea9-428c-9153-07f28bd9d48b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:18.598564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334710-4ea9-428c-9153-07f28bd9d48b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:18.599831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 440/858] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:18.600540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea334710-4ea9-428c-9153-07f28bd9d48b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:18.626293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 419/859] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Aug 29 23:31:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:18.627224 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8484d-b9c7-4e13-a12a-0ebbcea80f75 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:18.708018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:18.719217 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea334710-4ea9-428c-9153-07f28bd9d48b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:18.760769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947c41a-4923-4188-b5bf-a9e5d32ed71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:18.761385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947c41a-4923-4188-b5bf-a9e5d32ed71f None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:18.761663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947c41a-4923-4188-b5bf-a9e5d32ed71f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:18.761935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947c41a-4923-4188-b5bf-a9e5d32ed71f None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:18.762627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947c41a-4923-4188-b5bf-a9e5d32ed71f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:18.763491 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8484d-b9c7-4e13-a12a-0ebbcea80f75 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:18.764080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947c41a-4923-4188-b5bf-a9e5d32ed71f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:18.804197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151255f5-c7c9-403d-bc4f-1b228b6283d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:18.804861 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151255f5-c7c9-403d-bc4f-1b228b6283d3 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:18.805107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151255f5-c7c9-403d-bc4f-1b228b6283d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:18.805376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151255f5-c7c9-403d-bc4f-1b228b6283d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:18.813815 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151255f5-c7c9-403d-bc4f-1b228b6283d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:18.816356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151255f5-c7c9-403d-bc4f-1b228b6283d3 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-151255f5-c7c9-403d-bc4f-1b228b6283d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:18.828996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:18.830210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:18.832347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:18.832347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:18.832347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:18.832347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:18.832347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:18.832347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:18.832347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:18.832347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:18.832347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 420/860] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:18.832347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fde68-9004-4e8e-b5ae-53a3920881b3 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:18.832347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fde68-9004-4e8e-b5ae-53a3920881b3 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:18.833591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fde68-9004-4e8e-b5ae-53a3920881b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:18.847415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 441/861] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:31:18 2022] POST /identity/v3/services => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:31:18.848794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947c41a-4923-4188-b5bf-a9e5d32ed71f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:18.865465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-137fde68-9004-4e8e-b5ae-53a3920881b3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:18.867169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-137fde68-9004-4e8e-b5ae-53a3920881b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:18.907965 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:18.979695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 421/862] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:18 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:18.980494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fde68-9004-4e8e-b5ae-53a3920881b3 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:19.050088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:19.067378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947c41a-4923-4188-b5bf-a9e5d32ed71f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:19.136562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787f46b-7273-4914-bcc4-35026f92a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:19.137199 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787f46b-7273-4914-bcc4-35026f92a7fd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:19.137487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787f46b-7273-4914-bcc4-35026f92a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:19.137804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787f46b-7273-4914-bcc4-35026f92a7fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:19.165414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fde68-9004-4e8e-b5ae-53a3920881b3 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:19.238009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8ad76-d189-44d7-a32c-55e3682a2cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:19.239023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8ad76-d189-44d7-a32c-55e3682a2cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:19.239587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8ad76-d189-44d7-a32c-55e3682a2cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:19.240135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8ad76-d189-44d7-a32c-55e3682a2cb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:19.241843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8ad76-d189-44d7-a32c-55e3682a2cb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:19.242525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8ad76-d189-44d7-a32c-55e3682a2cb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:19.245440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8ad76-d189-44d7-a32c-55e3682a2cb5 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:19.271652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 422/863] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:19.281919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787f46b-7273-4914-bcc4-35026f92a7fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:19.282759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8ad76-d189-44d7-a32c-55e3682a2cb5 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:19.284272 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787f46b-7273-4914-bcc4-35026f92a7fd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:19.361085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:19.490467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 442/864] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:31:18 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:31:19.491735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787f46b-7273-4914-bcc4-35026f92a7fd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:19.504306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8ad76-d189-44d7-a32c-55e3682a2cb5 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:19.508313 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:19.572279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e4e9b-c99a-4645-9d4c-2e534cb9e249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:19.573705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e4e9b-c99a-4645-9d4c-2e534cb9e249 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:19.574199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e4e9b-c99a-4645-9d4c-2e534cb9e249 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:19.574726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e4e9b-c99a-4645-9d4c-2e534cb9e249 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:19.576127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e4e9b-c99a-4645-9d4c-2e534cb9e249 None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=c6b37b09a1ae45e8bc16adfa59500eb9) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:19.576836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e4e9b-c99a-4645-9d4c-2e534cb9e249 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:19.578857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e4e9b-c99a-4645-9d4c-2e534cb9e249 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:19.589851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 423/865] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:31:19 2022] GET /identity/v3/role_assignments?user.id=c6b37b09a1ae45e8bc16adfa59500eb9&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=d86dba8efe20423abdde06031690beb8 => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:19.591459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e4e9b-c99a-4645-9d4c-2e534cb9e249 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:19.671281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787f46b-7273-4914-bcc4-35026f92a7fd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:19.753424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ed71-59f6-40e7-bfab-5d0c51bffe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:19.754557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ed71-59f6-40e7-bfab-5d0c51bffe4e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:19.755058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ed71-59f6-40e7-bfab-5d0c51bffe4e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:19.755503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ed71-59f6-40e7-bfab-5d0c51bffe4e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:19.756593 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881ed71-59f6-40e7-bfab-5d0c51bffe4e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:19.759373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881ed71-59f6-40e7-bfab-5d0c51bffe4e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:19.763830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a881ed71-59f6-40e7-bfab-5d0c51bffe4e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:19.785531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:19.787516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 23:31:19.789265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:19.803012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 443/866] 10.222.0.58 () {56 vars in 1132 bytes} [Mon Aug 29 23:31:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:19.809746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ed71-59f6-40e7-bfab-5d0c51bffe4e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:19.818020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e4e9b-c99a-4645-9d4c-2e534cb9e249 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:19.872475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f246ee5-ee2f-4b74-a242-97be148778aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:19.874679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f246ee5-ee2f-4b74-a242-97be148778aa None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:19.875502 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f246ee5-ee2f-4b74-a242-97be148778aa None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:19.875996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f246ee5-ee2f-4b74-a242-97be148778aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:19.902168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f246ee5-ee2f-4b74-a242-97be148778aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:19.906352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f246ee5-ee2f-4b74-a242-97be148778aa None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-8f246ee5-ee2f-4b74-a242-97be148778aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:19.926249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:19.927285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:19.928404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:19.928404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:19.928404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:19.928404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 424/867] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:19.940878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f246ee5-ee2f-4b74-a242-97be148778aa None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:19.964875 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:19.991676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ed71-59f6-40e7-bfab-5d0c51bffe4e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:20.040239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e173381-36ee-40be-a2e1-67fed4ab861f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:20.041126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e173381-36ee-40be-a2e1-67fed4ab861f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:20.041559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e173381-36ee-40be-a2e1-67fed4ab861f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:20.041889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e173381-36ee-40be-a2e1-67fed4ab861f None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:20.042635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e173381-36ee-40be-a2e1-67fed4ab861f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:20.043016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e173381-36ee-40be-a2e1-67fed4ab861f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:20.044901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e173381-36ee-40be-a2e1-67fed4ab861f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:20.052501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 444/868] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Aug 29 23:31:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:20.062057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e173381-36ee-40be-a2e1-67fed4ab861f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:20.107753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f246ee5-ee2f-4b74-a242-97be148778aa None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:20.150832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:20.156520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ffe57-a537-4776-97dc-1cbbc83f6f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:20.157344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ffe57-a537-4776-97dc-1cbbc83f6f42 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:20.157700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ffe57-a537-4776-97dc-1cbbc83f6f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:20.158072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ffe57-a537-4776-97dc-1cbbc83f6f42 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:20.159039 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ffe57-a537-4776-97dc-1cbbc83f6f42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:20.159548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ffe57-a537-4776-97dc-1cbbc83f6f42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:20.161174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ffe57-a537-4776-97dc-1cbbc83f6f42 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:20.181397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 425/869] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:20.182387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ffe57-a537-4776-97dc-1cbbc83f6f42 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:20.210532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e173381-36ee-40be-a2e1-67fed4ab861f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:20.259113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79d4a6-967e-410f-b24f-006ebdf289be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:20.259832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79d4a6-967e-410f-b24f-006ebdf289be None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:20.260202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79d4a6-967e-410f-b24f-006ebdf289be None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:20.260546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79d4a6-967e-410f-b24f-006ebdf289be None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:20.261262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79d4a6-967e-410f-b24f-006ebdf289be None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:20.262354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79d4a6-967e-410f-b24f-006ebdf289be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:20.264041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79d4a6-967e-410f-b24f-006ebdf289be None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:20.274700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 445/870] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Aug 29 23:31:20 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:20.326171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:20.368309 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ffe57-a537-4776-97dc-1cbbc83f6f42 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:20.440476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ea4e4-1350-4717-941c-6af97d450d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:20.441488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ea4e4-1350-4717-941c-6af97d450d4c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:20.441951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ea4e4-1350-4717-941c-6af97d450d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:20.442449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ea4e4-1350-4717-941c-6af97d450d4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:20.458056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ea4e4-1350-4717-941c-6af97d450d4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:20.460832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ea4e4-1350-4717-941c-6af97d450d4c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-0d9ea4e4-1350-4717-941c-6af97d450d4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:20.485684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:20.486698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:20.487622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:20.487622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:20.487622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:20.487622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:20.487622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:20.487622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:20.487622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:20.487622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:20.488216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 426/871] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:20.495014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79d4a6-967e-410f-b24f-006ebdf289be None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:20.683198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79d4a6-967e-410f-b24f-006ebdf289be None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:20.734007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:20.749778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813026e-846e-4e5a-95a6-9e9d6a611574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:20.750838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813026e-846e-4e5a-95a6-9e9d6a611574 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:20.751315 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813026e-846e-4e5a-95a6-9e9d6a611574 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:20.751782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813026e-846e-4e5a-95a6-9e9d6a611574 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:20.752988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813026e-846e-4e5a-95a6-9e9d6a611574 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:20.753497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813026e-846e-4e5a-95a6-9e9d6a611574 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:20.756165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813026e-846e-4e5a-95a6-9e9d6a611574 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:20.782133 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 446/872] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:20.788354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ea4e4-1350-4717-941c-6af97d450d4c None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:20.943660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ea4e4-1350-4717-941c-6af97d450d4c None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:21.011294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525964de-3ebe-403c-961c-c36643b1cd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:21.012242 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525964de-3ebe-403c-961c-c36643b1cd1f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:21.012733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525964de-3ebe-403c-961c-c36643b1cd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:21.013217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525964de-3ebe-403c-961c-c36643b1cd1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:21.014869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525964de-3ebe-403c-961c-c36643b1cd1f None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=ff5b4f2f41604a8286141bce23db35c5) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:21.015422 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525964de-3ebe-403c-961c-c36643b1cd1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:21.018198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525964de-3ebe-403c-961c-c36643b1cd1f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:21.028860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 427/873] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:31:20 2022] GET /identity/v3/role_assignments?user.id=ff5b4f2f41604a8286141bce23db35c5&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:21.244501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9417878-d753-49c8-af46-32f3aa9c3568 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:21.244764 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9417878-d753-49c8-af46-32f3aa9c3568 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:21.245033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9417878-d753-49c8-af46-32f3aa9c3568 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:21.246742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 447/874] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:21.252876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbef68-eed0-473d-a106-82bb08b114f3 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:21.253398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbef68-eed0-473d-a106-82bb08b114f3 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:21.253398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbef68-eed0-473d-a106-82bb08b114f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:21.259920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:21.277176 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-02bbef68-eed0-473d-a106-82bb08b114f3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:21.279298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-02bbef68-eed0-473d-a106-82bb08b114f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:21.419150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 428/875] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:21.690049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9417878-d753-49c8-af46-32f3aa9c3568 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:21.831531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9417878-d753-49c8-af46-32f3aa9c3568 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:21.902072 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501213b7-40b1-4453-8727-f93609037002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:21.903177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501213b7-40b1-4453-8727-f93609037002 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:21.903635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501213b7-40b1-4453-8727-f93609037002 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:21.904078 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501213b7-40b1-4453-8727-f93609037002 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:21.905855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501213b7-40b1-4453-8727-f93609037002 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:21.908427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501213b7-40b1-4453-8727-f93609037002 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-501213b7-40b1-4453-8727-f93609037002 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:21.925707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:21.926843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 23:31:21.927695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:21.928432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 448/876] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:31:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:21.933426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbef68-eed0-473d-a106-82bb08b114f3 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:22.051106 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbef68-eed0-473d-a106-82bb08b114f3 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:22.087668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:22.092951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ab816-8cf9-4c6f-a366-0f5ad4674ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:22.093588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab816-8cf9-4c6f-a366-0f5ad4674ecb None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:22.093862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab816-8cf9-4c6f-a366-0f5ad4674ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:22.094130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab816-8cf9-4c6f-a366-0f5ad4674ecb None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:22.094822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ab816-8cf9-4c6f-a366-0f5ad4674ecb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:22.095119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ab816-8cf9-4c6f-a366-0f5ad4674ecb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:22.096533 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ab816-8cf9-4c6f-a366-0f5ad4674ecb None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:22.102133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 429/877] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:31:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:22.109250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501213b7-40b1-4453-8727-f93609037002 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:22.177901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:22.237620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501213b7-40b1-4453-8727-f93609037002 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:22.298933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc2bfc-f69f-47c5-b857-b4632be96532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:22.299968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc2bfc-f69f-47c5-b857-b4632be96532 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:22.300443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc2bfc-f69f-47c5-b857-b4632be96532 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:22.300984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc2bfc-f69f-47c5-b857-b4632be96532 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:22.302263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc2bfc-f69f-47c5-b857-b4632be96532 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:22.302748 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc2bfc-f69f-47c5-b857-b4632be96532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:22.305293 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc2bfc-f69f-47c5-b857-b4632be96532 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:22.315937 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 449/878] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:31:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:22.321490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ab816-8cf9-4c6f-a366-0f5ad4674ecb None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:22.473178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8c394-54c3-4d3c-8ec8-9651b70b7992 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:22.473517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8c394-54c3-4d3c-8ec8-9651b70b7992 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:22.473765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8c394-54c3-4d3c-8ec8-9651b70b7992 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:22.475434 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 450/879] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:31:22.480983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4a2ca-a98e-4d18-8bf3-bd1bc247702b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:22.481266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4a2ca-a98e-4d18-8bf3-bd1bc247702b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:22.481533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4a2ca-a98e-4d18-8bf3-bd1bc247702b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:22.503975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-84c4a2ca-a98e-4d18-8bf3-bd1bc247702b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:22.505687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-84c4a2ca-a98e-4d18-8bf3-bd1bc247702b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:22.518208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ab816-8cf9-4c6f-a366-0f5ad4674ecb None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:22.580850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1455f-6f80-4373-ad3d-c2eb6c3a4281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:22.581763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1455f-6f80-4373-ad3d-c2eb6c3a4281 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:22.582273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1455f-6f80-4373-ad3d-c2eb6c3a4281 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:22.582787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1455f-6f80-4373-ad3d-c2eb6c3a4281 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:22.584141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1455f-6f80-4373-ad3d-c2eb6c3a4281 None admin] RBAC: Adding query filter params (interface=public, service_id=a92cd19ad5b64b7bafed9cf1a244dcbb, region_id=RegionOne) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:22.584699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1455f-6f80-4373-ad3d-c2eb6c3a4281 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:22.587551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1455f-6f80-4373-ad3d-c2eb6c3a4281 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:22.594737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 430/880] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Aug 29 23:31:22 2022] GET /identity/v3/endpoints?service_id=a92cd19ad5b64b7bafed9cf1a244dcbb&interface=public®ion_id=RegionOne => generated 194 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:22.602496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1455f-6f80-4373-ad3d-c2eb6c3a4281 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:22.626824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 451/881] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:22 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:22.675443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:22.708282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2571690-3f04-4993-b9c3-c4e69c28ffca None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:22.709802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2571690-3f04-4993-b9c3-c4e69c28ffca None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:22.710596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2571690-3f04-4993-b9c3-c4e69c28ffca None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:22.713838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 452/882] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:22.725570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11847f39-457a-4ad4-8b15-b08ebaf8005b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:22.726370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1455f-6f80-4373-ad3d-c2eb6c3a4281 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:22.726831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11847f39-457a-4ad4-8b15-b08ebaf8005b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:22.727578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11847f39-457a-4ad4-8b15-b08ebaf8005b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:22.761069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-11847f39-457a-4ad4-8b15-b08ebaf8005b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:22.764297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-11847f39-457a-4ad4-8b15-b08ebaf8005b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:22.774546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e36fb2-f94a-4dd4-a7f5-b5ab39a71739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:22.775169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e36fb2-f94a-4dd4-a7f5-b5ab39a71739 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:22.775449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e36fb2-f94a-4dd4-a7f5-b5ab39a71739 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:22.775752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e36fb2-f94a-4dd4-a7f5-b5ab39a71739 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:22.776334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e36fb2-f94a-4dd4-a7f5-b5ab39a71739 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:22.777726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e36fb2-f94a-4dd4-a7f5-b5ab39a71739 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:22.784817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 431/883] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Aug 29 23:31:22 2022] GET /identity/v3/services => generated 1011 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:31:22.834982 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:22.877215 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 453/884] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:22 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:23.030933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e36fb2-f94a-4dd4-a7f5-b5ab39a71739 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:23.212404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e36fb2-f94a-4dd4-a7f5-b5ab39a71739 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:23.257154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4625aae-2aab-4e30-bc6f-7fdf3b3b2738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:23.257771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4625aae-2aab-4e30-bc6f-7fdf3b3b2738 None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:23.258055 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4625aae-2aab-4e30-bc6f-7fdf3b3b2738 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:23.258327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4625aae-2aab-4e30-bc6f-7fdf3b3b2738 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:23.259481 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4625aae-2aab-4e30-bc6f-7fdf3b3b2738 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:23.262637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:23.263222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4625aae-2aab-4e30-bc6f-7fdf3b3b2738 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:23.348440 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11847f39-457a-4ad4-8b15-b08ebaf8005b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:23.371138 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 432/885] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:31:23 2022] POST /identity/v3/services => generated 269 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:31:23.522498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11847f39-457a-4ad4-8b15-b08ebaf8005b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:23.554349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856224b7-c2e4-4c72-a51d-1372a6569d6e None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:23.554718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856224b7-c2e4-4c72-a51d-1372a6569d6e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:23.554989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856224b7-c2e4-4c72-a51d-1372a6569d6e None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:23.557569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 433/886] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:23.565655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2703a6-a725-4d6a-9d0b-075abc830eaa None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:23.566069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2703a6-a725-4d6a-9d0b-075abc830eaa None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:23.566485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2703a6-a725-4d6a-9d0b-075abc830eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:23.581250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4deff94-1fb0-4164-8977-db90f2a5ce83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:23.581914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4deff94-1fb0-4164-8977-db90f2a5ce83 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:23.582193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4deff94-1fb0-4164-8977-db90f2a5ce83 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:23.582536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4deff94-1fb0-4164-8977-db90f2a5ce83 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:23.583153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4deff94-1fb0-4164-8977-db90f2a5ce83 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:23.584600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4deff94-1fb0-4164-8977-db90f2a5ce83 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:23.593052 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-fc2703a6-a725-4d6a-9d0b-075abc830eaa None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:23.594832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-fc2703a6-a725-4d6a-9d0b-075abc830eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e4deff94-1fb0-4164-8977-db90f2a5ce83 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:23.595112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:23.596741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:23.598097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 454/887] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Aug 29 23:31:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:23.604847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4deff94-1fb0-4164-8977-db90f2a5ce83 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:23.665623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:23.722869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 434/888] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:23.808930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4deff94-1fb0-4164-8977-db90f2a5ce83 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:23.874270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e4174-2b8f-43a7-9619-ce61338a15d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:23.875643 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4174-2b8f-43a7-9619-ce61338a15d0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:23.876195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4174-2b8f-43a7-9619-ce61338a15d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:23.876734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4174-2b8f-43a7-9619-ce61338a15d0 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:23.877818 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e4174-2b8f-43a7-9619-ce61338a15d0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:23.878495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e4174-2b8f-43a7-9619-ce61338a15d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:23.880659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e4174-2b8f-43a7-9619-ce61338a15d0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:23.888758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 455/889] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Aug 29 23:31:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:23.895879 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2703a6-a725-4d6a-9d0b-075abc830eaa None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:24.016926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2703a6-a725-4d6a-9d0b-075abc830eaa None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:24.047704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:24.077916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e4174-2b8f-43a7-9619-ce61338a15d0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:24.084535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c80be-514a-4c71-a96e-afcd85bf7a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:24.084817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c80be-514a-4c71-a96e-afcd85bf7a11 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:24.085002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c80be-514a-4c71-a96e-afcd85bf7a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:24.085155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c80be-514a-4c71-a96e-afcd85bf7a11 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:24.085845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c80be-514a-4c71-a96e-afcd85bf7a11 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:24.086230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c80be-514a-4c71-a96e-afcd85bf7a11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:24.087768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c80be-514a-4c71-a96e-afcd85bf7a11 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:24.093965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 435/890] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Aug 29 23:31:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:24.274596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:24.280787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e4174-2b8f-43a7-9619-ce61338a15d0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:24.335007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1442-a21f-49bf-a031-11713441f37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:24.336100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1442-a21f-49bf-a031-11713441f37b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:24.336815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1442-a21f-49bf-a031-11713441f37b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:24.337585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1442-a21f-49bf-a031-11713441f37b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:24.363963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe1442-a21f-49bf-a031-11713441f37b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-2ffe1442-a21f-49bf-a031-11713441f37b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:24.369757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:24.371980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:24.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 456/891] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Aug 29 23:31:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:24.379047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c80be-514a-4c71-a96e-afcd85bf7a11 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:24.576918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c80be-514a-4c71-a96e-afcd85bf7a11 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:24.652448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210f1b7-86c1-4495-8b17-3fa06f3da00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:24.653401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210f1b7-86c1-4495-8b17-3fa06f3da00d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:24.653819 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210f1b7-86c1-4495-8b17-3fa06f3da00d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:24.654787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210f1b7-86c1-4495-8b17-3fa06f3da00d None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:24.656042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210f1b7-86c1-4495-8b17-3fa06f3da00d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:24.656561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210f1b7-86c1-4495-8b17-3fa06f3da00d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:24.659146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210f1b7-86c1-4495-8b17-3fa06f3da00d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:24.674412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 436/892] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Aug 29 23:31:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:24.681557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1442-a21f-49bf-a031-11713441f37b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:24.810947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1442-a21f-49bf-a031-11713441f37b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:24.853658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a8f7f-b8b8-4ac8-bff5-703c1012e82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:24.854391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a8f7f-b8b8-4ac8-bff5-703c1012e82f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:24.854696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a8f7f-b8b8-4ac8-bff5-703c1012e82f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:24.855012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a8f7f-b8b8-4ac8-bff5-703c1012e82f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:24.868703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a8f7f-b8b8-4ac8-bff5-703c1012e82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:24.870192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a8f7f-b8b8-4ac8-bff5-703c1012e82f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-7c9a8f7f-b8b8-4ac8-bff5-703c1012e82f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:24.886591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:24.889429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:24.893067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:24.893067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:24.893067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:24.893067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 457/893] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:24.896012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210f1b7-86c1-4495-8b17-3fa06f3da00d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:24.947322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:25.009408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ef3fc-fe80-463e-8119-fcae91cfac3a None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:25.010015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ef3fc-fe80-463e-8119-fcae91cfac3a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:25.010963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ef3fc-fe80-463e-8119-fcae91cfac3a None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:25.013226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 458/894] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:31:25.018792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210f1b7-86c1-4495-8b17-3fa06f3da00d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:25.019166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5c689-d8fa-4dec-99ef-89de76891075 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:25.019451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5c689-d8fa-4dec-99ef-89de76891075 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:25.019721 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5c689-d8fa-4dec-99ef-89de76891075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:25.039620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4dd5c689-d8fa-4dec-99ef-89de76891075 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:25.040955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4dd5c689-d8fa-4dec-99ef-89de76891075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:25.046345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:25.071919 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ef208-f12d-41ea-bd90-1131af165f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:25.073301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ef208-f12d-41ea-bd90-1131af165f94 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:25.073807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ef208-f12d-41ea-bd90-1131af165f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:25.074504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ef208-f12d-41ea-bd90-1131af165f94 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:25.075820 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ef208-f12d-41ea-bd90-1131af165f94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:25.077496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ef208-f12d-41ea-bd90-1131af165f94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:25.080091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ef208-f12d-41ea-bd90-1131af165f94 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:25.103790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 437/895] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:25.105121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ef208-f12d-41ea-bd90-1131af165f94 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:25.133375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 459/896] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:25.228767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:25.233140 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ef208-f12d-41ea-bd90-1131af165f94 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:25.273285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a383b-89e7-4d79-8896-10803dd29749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:25.274318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a383b-89e7-4d79-8896-10803dd29749 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:25.274875 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a383b-89e7-4d79-8896-10803dd29749 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:25.275365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a383b-89e7-4d79-8896-10803dd29749 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:25.284736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a383b-89e7-4d79-8896-10803dd29749 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:25.286692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a383b-89e7-4d79-8896-10803dd29749 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-db9a383b-89e7-4d79-8896-10803dd29749 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:25.301396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:25.302463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:25.303448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:25.303448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:25.303448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:25.303448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:25.303448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:25.303448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:25.303448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:25.303448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:25.303448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:31:25.303448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:25.306132 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 438/897] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Aug 29 23:31:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:25.313470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5c689-d8fa-4dec-99ef-89de76891075 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:25.415191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a383b-89e7-4d79-8896-10803dd29749 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:25.432549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5c689-d8fa-4dec-99ef-89de76891075 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:25.472520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f16d2-90a3-4a7c-9c75-1f397a4535cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:25.473135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f16d2-90a3-4a7c-9c75-1f397a4535cc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:25.473481 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f16d2-90a3-4a7c-9c75-1f397a4535cc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:25.473757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f16d2-90a3-4a7c-9c75-1f397a4535cc None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:25.474520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f16d2-90a3-4a7c-9c75-1f397a4535cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:25.474828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f16d2-90a3-4a7c-9c75-1f397a4535cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:25.476556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f16d2-90a3-4a7c-9c75-1f397a4535cc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:25.490530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:25.502468 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 460/898] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Aug 29 23:31:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:25.509145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f16d2-90a3-4a7c-9c75-1f397a4535cc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:25.570317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a383b-89e7-4d79-8896-10803dd29749 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:25.611532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f38491-0238-4743-adb8-74b95bd2d017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:25.612190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f38491-0238-4743-adb8-74b95bd2d017 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:25.612465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f38491-0238-4743-adb8-74b95bd2d017 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:25.612733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f38491-0238-4743-adb8-74b95bd2d017 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:25.613328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f38491-0238-4743-adb8-74b95bd2d017 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:25.614733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f38491-0238-4743-adb8-74b95bd2d017 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-02f38491-0238-4743-adb8-74b95bd2d017 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:25.625614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:25.626692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 23:31:25.627536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:25.630737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 439/899] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:31:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:25.633423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f38491-0238-4743-adb8-74b95bd2d017 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:25.653708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f16d2-90a3-4a7c-9c75-1f397a4535cc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:25.674199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:25.714556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51bb36-a0dc-471e-958b-73b0640529d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:25.715768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51bb36-a0dc-471e-958b-73b0640529d0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:25.717298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51bb36-a0dc-471e-958b-73b0640529d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:25.717769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51bb36-a0dc-471e-958b-73b0640529d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:25.812536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f38491-0238-4743-adb8-74b95bd2d017 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:25.818949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:25.823806 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51bb36-a0dc-471e-958b-73b0640529d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:25.825617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51bb36-a0dc-471e-958b-73b0640529d0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:25.881202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1c6cf-e36c-400c-8534-fcdbb5611df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:25.882857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c6cf-e36c-400c-8534-fcdbb5611df1 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:25.883064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c6cf-e36c-400c-8534-fcdbb5611df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:25.883547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c6cf-e36c-400c-8534-fcdbb5611df1 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:25.884853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1c6cf-e36c-400c-8534-fcdbb5611df1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:25.885411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1c6cf-e36c-400c-8534-fcdbb5611df1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:25.888268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1c6cf-e36c-400c-8534-fcdbb5611df1 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:25.899018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 440/900] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:31:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:25.907215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1c6cf-e36c-400c-8534-fcdbb5611df1 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:25.973394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 461/901] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:31:25 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:31:25.976234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51bb36-a0dc-471e-958b-73b0640529d0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:26.097558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:26.113411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1c6cf-e36c-400c-8534-fcdbb5611df1 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:26.147644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:26.153049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51bb36-a0dc-471e-958b-73b0640529d0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:26.158605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12931856-cb1e-41b8-b705-f4d4fc1deb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:26.159644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12931856-cb1e-41b8-b705-f4d4fc1deb59 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:26.159644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12931856-cb1e-41b8-b705-f4d4fc1deb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:26.159914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12931856-cb1e-41b8-b705-f4d4fc1deb59 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:26.160618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12931856-cb1e-41b8-b705-f4d4fc1deb59 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:26.160970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12931856-cb1e-41b8-b705-f4d4fc1deb59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:26.162394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12931856-cb1e-41b8-b705-f4d4fc1deb59 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:26.171118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 441/902] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Aug 29 23:31:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:26.173202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084645b-4ece-4d63-9e8e-8e569d6e88fa None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:26.173676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084645b-4ece-4d63-9e8e-8e569d6e88fa None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:26.174136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084645b-4ece-4d63-9e8e-8e569d6e88fa None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:26.179404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 442/903] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:26.180666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084645b-4ece-4d63-9e8e-8e569d6e88fa None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:26.215598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7635d-d12b-4e31-8fa1-1791a656e50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:26.216838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7635d-d12b-4e31-8fa1-1791a656e50d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:26.217335 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7635d-d12b-4e31-8fa1-1791a656e50d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:26.217818 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7635d-d12b-4e31-8fa1-1791a656e50d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:26.241057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7635d-d12b-4e31-8fa1-1791a656e50d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:26.244406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7635d-d12b-4e31-8fa1-1791a656e50d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-4cf7635d-d12b-4e31-8fa1-1791a656e50d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:26.262976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:26.266687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:26.268414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:26.268414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:26.268414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:26.268414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 462/904] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:26.268414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdfdec-ab10-4488-b357-377eeda9c3af None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:26.268414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdfdec-ab10-4488-b357-377eeda9c3af None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:26.269055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdfdec-ab10-4488-b357-377eeda9c3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:26.291354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-34fdfdec-ab10-4488-b357-377eeda9c3af None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:26.292897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-34fdfdec-ab10-4488-b357-377eeda9c3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:26.352090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084645b-4ece-4d63-9e8e-8e569d6e88fa None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:26.402464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 463/905] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:26.402464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdfdec-ab10-4488-b357-377eeda9c3af None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:26.406215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25925269-84a9-434b-8db5-2ca4698751b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:26.407013 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25925269-84a9-434b-8db5-2ca4698751b3 None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:26.407364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25925269-84a9-434b-8db5-2ca4698751b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:26.407702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25925269-84a9-434b-8db5-2ca4698751b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:26.408484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25925269-84a9-434b-8db5-2ca4698751b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:26.409998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25925269-84a9-434b-8db5-2ca4698751b3 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:26.546954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 443/906] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:31:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:31:26.565765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:26.607920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdfdec-ab10-4488-b357-377eeda9c3af None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:26.681655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8f58c-a980-4a2a-8997-3486e5920475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:26.682274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8f58c-a980-4a2a-8997-3486e5920475 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:26.682587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8f58c-a980-4a2a-8997-3486e5920475 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:26.682859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8f58c-a980-4a2a-8997-3486e5920475 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:26.683609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8f58c-a980-4a2a-8997-3486e5920475 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:26.683980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8f58c-a980-4a2a-8997-3486e5920475 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:26.685983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8f58c-a980-4a2a-8997-3486e5920475 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:26.699686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 464/907] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:26.707255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25925269-84a9-434b-8db5-2ca4698751b3 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:26.729257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:26.748241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8f58c-a980-4a2a-8997-3486e5920475 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:26.877985 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25925269-84a9-434b-8db5-2ca4698751b3 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:26.893784 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8f58c-a980-4a2a-8997-3486e5920475 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:26.924306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325db3f5-5868-478d-9d1a-84a2cce32a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:26.924958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db3f5-5868-478d-9d1a-84a2cce32a2a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:26.925239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db3f5-5868-478d-9d1a-84a2cce32a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:26.925508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db3f5-5868-478d-9d1a-84a2cce32a2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:26.933472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4811166-c412-4cf9-880f-1740ed1b5554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:26.933739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325db3f5-5868-478d-9d1a-84a2cce32a2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:26.934105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4811166-c412-4cf9-880f-1740ed1b5554 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:26.934382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4811166-c412-4cf9-880f-1740ed1b5554 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:26.934720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4811166-c412-4cf9-880f-1740ed1b5554 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:26.936017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325db3f5-5868-478d-9d1a-84a2cce32a2a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:26.936181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4811166-c412-4cf9-880f-1740ed1b5554 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:26.937228 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4811166-c412-4cf9-880f-1740ed1b5554 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:26.940323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e4811166-c412-4cf9-880f-1740ed1b5554 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:26.947494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:26.949915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:26.952513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 465/908] 10.222.0.58 () {56 vars in 1132 bytes} [Mon Aug 29 23:31:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:26.959733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa9cea-62d5-4870-b2d8-7d961f13e6c1 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:26.959733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa9cea-62d5-4870-b2d8-7d961f13e6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:26.959733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa9cea-62d5-4870-b2d8-7d961f13e6c1 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:26.959733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 466/909] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:26.959733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa9cea-62d5-4870-b2d8-7d961f13e6c1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-325db3f5-5868-478d-9d1a-84a2cce32a2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:26.965157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:26.966001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:26.967538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:26.967538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:26.967538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:26.967538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:26.967538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:26.967538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:26.967538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:26.967538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:26.967538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 444/910] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:26.967538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9284cdb-c929-470c-9238-6c731e9f7a36 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:26.967538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9284cdb-c929-470c-9238-6c731e9f7a36 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:26.968164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9284cdb-c929-470c-9238-6c731e9f7a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:26.992170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d9284cdb-c929-470c-9238-6c731e9f7a36 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:26.993511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d9284cdb-c929-470c-9238-6c731e9f7a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:27.077728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:27.084710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 445/911] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:27.086235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9284cdb-c929-470c-9238-6c731e9f7a36 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:27.098487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa9cea-62d5-4870-b2d8-7d961f13e6c1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:27.157650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683e2a0-b2e1-44dd-96fc-611ac6015559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:27.158641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683e2a0-b2e1-44dd-96fc-611ac6015559 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:27.159176 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683e2a0-b2e1-44dd-96fc-611ac6015559 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:27.159641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683e2a0-b2e1-44dd-96fc-611ac6015559 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:27.160463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683e2a0-b2e1-44dd-96fc-611ac6015559 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:27.161685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683e2a0-b2e1-44dd-96fc-611ac6015559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:27.163422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683e2a0-b2e1-44dd-96fc-611ac6015559 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:27.175921 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 467/912] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Aug 29 23:31:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:27.179111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683e2a0-b2e1-44dd-96fc-611ac6015559 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:27.240183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9284cdb-c929-470c-9238-6c731e9f7a36 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:27.276289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:27.281180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f146c-23fb-4338-afa9-4cc0add89eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:27.281807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f146c-23fb-4338-afa9-4cc0add89eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:27.282085 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f146c-23fb-4338-afa9-4cc0add89eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:27.282359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f146c-23fb-4338-afa9-4cc0add89eb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:27.283162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f146c-23fb-4338-afa9-4cc0add89eb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:27.283480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f146c-23fb-4338-afa9-4cc0add89eb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:27.285034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f146c-23fb-4338-afa9-4cc0add89eb3 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:27.298921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 446/913] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:27.307883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f146c-23fb-4338-afa9-4cc0add89eb3 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:27.330520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683e2a0-b2e1-44dd-96fc-611ac6015559 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:27.379302 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c730978-767b-4820-b2ff-c68003b32d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:27.379915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c730978-767b-4820-b2ff-c68003b32d3f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:27.380194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c730978-767b-4820-b2ff-c68003b32d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:27.380466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c730978-767b-4820-b2ff-c68003b32d3f None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:27.381100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c730978-767b-4820-b2ff-c68003b32d3f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:27.381400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c730978-767b-4820-b2ff-c68003b32d3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:27.383454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c730978-767b-4820-b2ff-c68003b32d3f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:27.397618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 468/914] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Aug 29 23:31:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:27.398290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c730978-767b-4820-b2ff-c68003b32d3f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:27.421709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:27.508145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f146c-23fb-4338-afa9-4cc0add89eb3 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:27.542236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c730978-767b-4820-b2ff-c68003b32d3f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:27.569848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf73481-aebf-4f65-8641-0514930b76f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:27.570898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf73481-aebf-4f65-8641-0514930b76f0 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:27.571187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf73481-aebf-4f65-8641-0514930b76f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:27.571455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf73481-aebf-4f65-8641-0514930b76f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:27.572479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf73481-aebf-4f65-8641-0514930b76f0 None admin] RBAC: Adding query filter params (role.id=c6a4683645ce4b68a14f2edeee414d92, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=ff5b4f2f41604a8286141bce23db35c5) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:27.572791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf73481-aebf-4f65-8641-0514930b76f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:27.574153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:27.575112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf73481-aebf-4f65-8641-0514930b76f0 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:27.584550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007d05d-9b75-4442-8bb0-6091829b42ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:27.591648 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 447/915] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:31:27 2022] GET /identity/v3/role_assignments?user.id=ff5b4f2f41604a8286141bce23db35c5&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=c6a4683645ce4b68a14f2edeee414d92 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:27.591830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007d05d-9b75-4442-8bb0-6091829b42ec None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:27.591830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007d05d-9b75-4442-8bb0-6091829b42ec None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:27.591830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007d05d-9b75-4442-8bb0-6091829b42ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:27.591830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007d05d-9b75-4442-8bb0-6091829b42ec None admin] RBAC: Adding query filter params (interface=public, service_id=26eacf14a2e64ae0a8bfe0cfa720a059, region_id=RegionOne) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:27.592134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf73481-aebf-4f65-8641-0514930b76f0 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:27.592214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007d05d-9b75-4442-8bb0-6091829b42ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:27.592214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007d05d-9b75-4442-8bb0-6091829b42ec None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:27.608994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 469/916] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Aug 29 23:31:27 2022] GET /identity/v3/endpoints?service_id=26eacf14a2e64ae0a8bfe0cfa720a059&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:27.610591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007d05d-9b75-4442-8bb0-6091829b42ec None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:27.793601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007d05d-9b75-4442-8bb0-6091829b42ec None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:27.824643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf73481-aebf-4f65-8641-0514930b76f0 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:27.861283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1d703-c91d-4d8a-85ab-e2557bb9d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:27.861819 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1d703-c91d-4d8a-85ab-e2557bb9d338 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:27.862194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1d703-c91d-4d8a-85ab-e2557bb9d338 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:27.862610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1d703-c91d-4d8a-85ab-e2557bb9d338 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:27.863315 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1d703-c91d-4d8a-85ab-e2557bb9d338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:27.864800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1d703-c91d-4d8a-85ab-e2557bb9d338 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:27.871843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 470/917] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Aug 29 23:31:27 2022] GET /identity/v3/services => generated 1268 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:31:27.885641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:27.910273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4dca4-bedf-490c-9916-7a885a1a81e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:27.911392 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4dca4-bedf-490c-9916-7a885a1a81e6 None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:27.912168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4dca4-bedf-490c-9916-7a885a1a81e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:27.912667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4dca4-bedf-490c-9916-7a885a1a81e6 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:27.913838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4dca4-bedf-490c-9916-7a885a1a81e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:27.916624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4dca4-bedf-490c-9916-7a885a1a81e6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:27.989896 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 448/918] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:31:27 2022] POST /identity/v3/services => generated 241 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:31:28.766849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bf562-3f26-43f3-a6a3-27d5bb570c55 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:28.767508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bf562-3f26-43f3-a6a3-27d5bb570c55 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:28.767508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bf562-3f26-43f3-a6a3-27d5bb570c55 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:28.768945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 471/919] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:28.775863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e927b0e-41f1-48ea-be1e-247c806ef68d None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:28.776525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e927b0e-41f1-48ea-be1e-247c806ef68d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:28.776525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e927b0e-41f1-48ea-be1e-247c806ef68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:28.801102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-3e927b0e-41f1-48ea-be1e-247c806ef68d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:28.803199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3e927b0e-41f1-48ea-be1e-247c806ef68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:28.949201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 449/920] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:29.234871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bf562-3f26-43f3-a6a3-27d5bb570c55 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:29.300566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:29.350578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bf562-3f26-43f3-a6a3-27d5bb570c55 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:29.390389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca6711-acf4-4baf-a619-5b10fa0cdd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:29.391051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca6711-acf4-4baf-a619-5b10fa0cdd59 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:29.391331 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca6711-acf4-4baf-a619-5b10fa0cdd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:29.391603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca6711-acf4-4baf-a619-5b10fa0cdd59 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:29.392269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca6711-acf4-4baf-a619-5b10fa0cdd59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:29.393715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca6711-acf4-4baf-a619-5b10fa0cdd59 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:29.396362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: WARNING keystone.server.flask.common [None req-3fca6711-acf4-4baf-a619-5b10fa0cdd59 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 23:31:29.452960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.sql.core [None req-3fca6711-acf4-4baf-a619-5b10fa0cdd59 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 23:31:29.452960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 23:31:29.452960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [parameters: {'id': 'f385cd4d426346b08324ab55d6cbeebd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 23:31:29.452960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81852) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-3fca6711-acf4-4baf-a619-5b10fa0cdd59 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f385cd4d426346b08324ab55d6cbeebd.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f385cd4d426346b08324ab55d6cbeebd. Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application conn.query(q) Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 23:31:29.486600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result.read() Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 29 23:31:29.489024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application next(self.gen) Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application yield resource Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application next(self.gen) Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise self.value Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application session.commit() Aug 29 23:31:29.496750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.session.flush() Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application compat.raise_( Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 23:31:29.500697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 23:31:29.504114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application conn.query(q) Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result.read() Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 23:31:29.507194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [parameters: {'id': 'f385cd4d426346b08324ab55d6cbeebd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:29.510603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:29.514201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:29.517860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:29.517860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 29 23:31:29.517860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 29 23:31:29.517860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:29.517860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:29.517860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 29 23:31:29.517860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 29 23:31:29.517860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f385cd4d426346b08324ab55d6cbeebd. Aug 29 23:31:29.517860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:29.517860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 472/921] 10.222.0.58 () {60 vars in 1140 bytes} [Mon Aug 29 23:31:29 2022] POST /identity/v3/projects => generated 262 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:29.527596 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e927b0e-41f1-48ea-be1e-247c806ef68d None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:29.701859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e927b0e-41f1-48ea-be1e-247c806ef68d None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:29.720008 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:29.766854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bbb7b-17dd-49df-aa70-d06fe31fb0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:29.767821 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bbb7b-17dd-49df-aa70-d06fe31fb0fe None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:29.768397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bbb7b-17dd-49df-aa70-d06fe31fb0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:29.768881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bbb7b-17dd-49df-aa70-d06fe31fb0fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:29.781863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bbb7b-17dd-49df-aa70-d06fe31fb0fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:29.784575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bbb7b-17dd-49df-aa70-d06fe31fb0fe None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-2c6bbb7b-17dd-49df-aa70-d06fe31fb0fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:29.807098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:29.810514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:29.813739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:29.813739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:29.813739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:29.813739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:29.813739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:29.813739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:29.813739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:29.813739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:29.813739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 450/922] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:29.820869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca6711-acf4-4baf-a619-5b10fa0cdd59 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:30.027534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca6711-acf4-4baf-a619-5b10fa0cdd59 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:30.055742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:30.097843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ddac4-1d25-466a-8c6d-3e7a48603a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:30.098951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ddac4-1d25-466a-8c6d-3e7a48603a45 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:30.099453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ddac4-1d25-466a-8c6d-3e7a48603a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:30.099937 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ddac4-1d25-466a-8c6d-3e7a48603a45 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:30.101190 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ddac4-1d25-466a-8c6d-3e7a48603a45 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:30.101727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ddac4-1d25-466a-8c6d-3e7a48603a45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:30.104966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ddac4-1d25-466a-8c6d-3e7a48603a45 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:30.149402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 473/923] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:31:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:30.242377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a982d03-3c27-47ea-bb67-d452b2ec1586 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:30.242872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a982d03-3c27-47ea-bb67-d452b2ec1586 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:30.243310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a982d03-3c27-47ea-bb67-d452b2ec1586 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:30.246623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 451/924] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:30.253080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfb2fd-7807-4329-9071-70fdba2d55e5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:30.253721 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfb2fd-7807-4329-9071-70fdba2d55e5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:30.254200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfb2fd-7807-4329-9071-70fdba2d55e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:30.276353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-30bfb2fd-7807-4329-9071-70fdba2d55e5 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:30.279059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-30bfb2fd-7807-4329-9071-70fdba2d55e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:30.292935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5d533-8149-49f1-bebc-1ea88bd3876b None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:30.293470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5d533-8149-49f1-bebc-1ea88bd3876b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:30.293973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5d533-8149-49f1-bebc-1ea88bd3876b None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:30.298550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 452/925] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:30.304021 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaae770-06be-4fa3-908d-f83e1a968563 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:30.304332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaae770-06be-4fa3-908d-f83e1a968563 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:30.304571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaae770-06be-4fa3-908d-f83e1a968563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:30.332618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-5eaae770-06be-4fa3-908d-f83e1a968563 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:30.335649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5eaae770-06be-4fa3-908d-f83e1a968563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:30.398053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 474/926] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:30.454756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 453/927] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:30.481436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcf32a-394d-4dad-9551-c796616ac676 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:30.481836 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcf32a-394d-4dad-9551-c796616ac676 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:30.482309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcf32a-394d-4dad-9551-c796616ac676 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:30.484909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 475/928] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:31:30.489573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362ac18-86e4-4395-b7b7-f09f9f4c47b7 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:30.490036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362ac18-86e4-4395-b7b7-f09f9f4c47b7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:30.490272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362ac18-86e4-4395-b7b7-f09f9f4c47b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:30.514999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-9362ac18-86e4-4395-b7b7-f09f9f4c47b7 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:30.516468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9362ac18-86e4-4395-b7b7-f09f9f4c47b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:30.591248 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 454/929] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:30.759540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcf32a-394d-4dad-9551-c796616ac676 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:30.856018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362ac18-86e4-4395-b7b7-f09f9f4c47b7 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:30.897308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcf32a-394d-4dad-9551-c796616ac676 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:30.899576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:30.909772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:30.943358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de419d-4e2c-4da9-bc1d-9001b7e95f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:30.944311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de419d-4e2c-4da9-bc1d-9001b7e95f19 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:30.944760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de419d-4e2c-4da9-bc1d-9001b7e95f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:30.945232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de419d-4e2c-4da9-bc1d-9001b7e95f19 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:30.962099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de419d-4e2c-4da9-bc1d-9001b7e95f19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-26de419d-4e2c-4da9-bc1d-9001b7e95f19 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:30.969145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:30.970151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:31:30.971530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:30.972161 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 476/930] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Aug 29 23:31:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:30.972802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de419d-4e2c-4da9-bc1d-9001b7e95f19 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:31.017654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362ac18-86e4-4395-b7b7-f09f9f4c47b7 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:31.070872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90fa8c-dda4-4ecf-a241-8fa874dda320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:31.071504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90fa8c-dda4-4ecf-a241-8fa874dda320 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:31.071784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90fa8c-dda4-4ecf-a241-8fa874dda320 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:31.072059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90fa8c-dda4-4ecf-a241-8fa874dda320 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:31.072669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90fa8c-dda4-4ecf-a241-8fa874dda320 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:31.074070 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90fa8c-dda4-4ecf-a241-8fa874dda320 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-4c90fa8c-dda4-4ecf-a241-8fa874dda320 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:31.084276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:31.085591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:31.086712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 455/931] 10.222.0.58 () {56 vars in 1132 bytes} [Mon Aug 29 23:31:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:31.087526 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90fa8c-dda4-4ecf-a241-8fa874dda320 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:31.132405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de419d-4e2c-4da9-bc1d-9001b7e95f19 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:31.178981 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c091e-7509-4449-afc7-598ddfce7046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:31.180051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c091e-7509-4449-afc7-598ddfce7046 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:31.180542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c091e-7509-4449-afc7-598ddfce7046 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:31.181041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c091e-7509-4449-afc7-598ddfce7046 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:31.182100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c091e-7509-4449-afc7-598ddfce7046 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:31.185194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c091e-7509-4449-afc7-598ddfce7046 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-8d2c091e-7509-4449-afc7-598ddfce7046 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:31.197022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:31.198566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:31.200187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 477/932] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Aug 29 23:31:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:31.203976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c091e-7509-4449-afc7-598ddfce7046 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:31.210893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:31.223566 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90fa8c-dda4-4ecf-a241-8fa874dda320 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:31.251669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:31.263877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b52f7-264a-496b-a88b-a4550e1bd287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:31.263877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b52f7-264a-496b-a88b-a4550e1bd287 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:31.264935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b52f7-264a-496b-a88b-a4550e1bd287 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:31.265333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b52f7-264a-496b-a88b-a4550e1bd287 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:31.265915 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b52f7-264a-496b-a88b-a4550e1bd287 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:31.266217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b52f7-264a-496b-a88b-a4550e1bd287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:31.267651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b52f7-264a-496b-a88b-a4550e1bd287 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:31.292225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 456/933] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Aug 29 23:31:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:31.292225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b52f7-264a-496b-a88b-a4550e1bd287 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:31.364390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c091e-7509-4449-afc7-598ddfce7046 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:31.408950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c7548-7482-49f4-bf18-ab8fbcf2e9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:31.409522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c7548-7482-49f4-bf18-ab8fbcf2e9cc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:31.409876 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c7548-7482-49f4-bf18-ab8fbcf2e9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:31.410232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c7548-7482-49f4-bf18-ab8fbcf2e9cc None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:31.410933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c7548-7482-49f4-bf18-ab8fbcf2e9cc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:31.411338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c7548-7482-49f4-bf18-ab8fbcf2e9cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:31.413116 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c7548-7482-49f4-bf18-ab8fbcf2e9cc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:31.422484 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 478/934] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Aug 29 23:31:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:31.422484 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c7548-7482-49f4-bf18-ab8fbcf2e9cc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:31.458092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b52f7-264a-496b-a88b-a4550e1bd287 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:31.511847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743effa-c7b5-4e4d-bee0-4bb270d12a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:31.512462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743effa-c7b5-4e4d-bee0-4bb270d12a69 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:31.512769 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743effa-c7b5-4e4d-bee0-4bb270d12a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:31.513730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743effa-c7b5-4e4d-bee0-4bb270d12a69 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:31.514708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743effa-c7b5-4e4d-bee0-4bb270d12a69 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:31.515268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743effa-c7b5-4e4d-bee0-4bb270d12a69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:31.516655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743effa-c7b5-4e4d-bee0-4bb270d12a69 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:31.523699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 457/935] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Aug 29 23:31:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:31.525128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743effa-c7b5-4e4d-bee0-4bb270d12a69 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:31.555243 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:31.568615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:31.581191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c7548-7482-49f4-bf18-ab8fbcf2e9cc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:31.620342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af217280-7cca-4a39-a756-91148442abd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:31.620988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217280-7cca-4a39-a756-91148442abd8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:31.621272 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217280-7cca-4a39-a756-91148442abd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:31.621542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217280-7cca-4a39-a756-91148442abd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:31.633406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af217280-7cca-4a39-a756-91148442abd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:31.634911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af217280-7cca-4a39-a756-91148442abd8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-af217280-7cca-4a39-a756-91148442abd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:31.651777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:31.653186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:31.655140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:31.655140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:31.655140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:31.655140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 479/936] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:31.655140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af217280-7cca-4a39-a756-91148442abd8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:31.663759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743effa-c7b5-4e4d-bee0-4bb270d12a69 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:31.705305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7f014-7c68-409d-9220-a9b9072abb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:31.706351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7f014-7c68-409d-9220-a9b9072abb4b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:31.706883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7f014-7c68-409d-9220-a9b9072abb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:31.707376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7f014-7c68-409d-9220-a9b9072abb4b None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:31.708451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7f014-7c68-409d-9220-a9b9072abb4b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:31.708766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7f014-7c68-409d-9220-a9b9072abb4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:31.711427 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7f014-7c68-409d-9220-a9b9072abb4b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:31.721063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 458/937] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Aug 29 23:31:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:31.722205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7f014-7c68-409d-9220-a9b9072abb4b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:31.822944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af217280-7cca-4a39-a756-91148442abd8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:31.856737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:31.869180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea72145-5868-48fd-b80d-186f0bf07241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:31.869809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea72145-5868-48fd-b80d-186f0bf07241 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:31.870087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea72145-5868-48fd-b80d-186f0bf07241 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:31.870361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea72145-5868-48fd-b80d-186f0bf07241 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:31.872934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea72145-5868-48fd-b80d-186f0bf07241 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:31.872934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea72145-5868-48fd-b80d-186f0bf07241 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:31.873475 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea72145-5868-48fd-b80d-186f0bf07241 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:31.884862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 480/938] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Aug 29 23:31:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:31.886140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea72145-5868-48fd-b80d-186f0bf07241 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:31.891114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7f014-7c68-409d-9220-a9b9072abb4b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:31.898102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:31.940871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e2be5-ed54-4712-91d6-650beb624168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:31.941405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e2be5-ed54-4712-91d6-650beb624168 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:31.941690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e2be5-ed54-4712-91d6-650beb624168 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:31.941968 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e2be5-ed54-4712-91d6-650beb624168 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:31.942738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e2be5-ed54-4712-91d6-650beb624168 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:31.942984 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e2be5-ed54-4712-91d6-650beb624168 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:31.944376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e2be5-ed54-4712-91d6-650beb624168 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:31.974642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 459/939] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:31.974642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e2be5-ed54-4712-91d6-650beb624168 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:32.059245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea72145-5868-48fd-b80d-186f0bf07241 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:32.093690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e2be5-ed54-4712-91d6-650beb624168 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:32.111480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0aaac-1074-4672-86c7-2ec20dca5448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:32.112366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0aaac-1074-4672-86c7-2ec20dca5448 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:32.112830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0aaac-1074-4672-86c7-2ec20dca5448 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:32.113233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0aaac-1074-4672-86c7-2ec20dca5448 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:32.114351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0aaac-1074-4672-86c7-2ec20dca5448 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:32.116215 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0aaac-1074-4672-86c7-2ec20dca5448 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:32.138753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a85cc9-e703-4a7f-b258-a1137fbe643d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:32.139631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a85cc9-e703-4a7f-b258-a1137fbe643d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:32.140228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a85cc9-e703-4a7f-b258-a1137fbe643d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:32.140549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a85cc9-e703-4a7f-b258-a1137fbe643d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:32.141331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a85cc9-e703-4a7f-b258-a1137fbe643d None admin] RBAC: Adding query filter params (interface=public, service_id=e2eee657072340efa0bff20c8e96634f, region_id=RegionOne) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:32.141634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a85cc9-e703-4a7f-b258-a1137fbe643d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:32.143042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a85cc9-e703-4a7f-b258-a1137fbe643d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:32.154303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 460/940] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Aug 29 23:31:31 2022] GET /identity/v3/endpoints?service_id=e2eee657072340efa0bff20c8e96634f&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:32.154303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a85cc9-e703-4a7f-b258-a1137fbe643d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:32.160129 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:32.228340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 481/941] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:31:31 2022] POST /identity/v3/endpoints => generated 347 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:31:32.239669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0aaac-1074-4672-86c7-2ec20dca5448 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:32.313788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a85cc9-e703-4a7f-b258-a1137fbe643d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:32.357012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:32.385365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be7265-680d-4e1b-8e97-3e5fd383f5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:32.386052 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be7265-680d-4e1b-8e97-3e5fd383f5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:32.386349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be7265-680d-4e1b-8e97-3e5fd383f5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:32.386620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be7265-680d-4e1b-8e97-3e5fd383f5e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:32.396333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be7265-680d-4e1b-8e97-3e5fd383f5e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:32.397765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be7265-680d-4e1b-8e97-3e5fd383f5e9 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-e7be7265-680d-4e1b-8e97-3e5fd383f5e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:32.410949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:32.412001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:32.413134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:32.413134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:32.413134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:32.413134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:32.413134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:32.413134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:32.413134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:32.413134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:32.413134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:31:32.413134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:32.414588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 461/942] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Aug 29 23:31:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:32.419387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be7265-680d-4e1b-8e97-3e5fd383f5e9 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:32.428941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0aaac-1074-4672-86c7-2ec20dca5448 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:32.484831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b40de-bca7-48eb-b30f-3733137c19cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:32.485401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b40de-bca7-48eb-b30f-3733137c19cf None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:32.485689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b40de-bca7-48eb-b30f-3733137c19cf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:32.485963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b40de-bca7-48eb-b30f-3733137c19cf None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:32.487742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b40de-bca7-48eb-b30f-3733137c19cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:32.490491 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b40de-bca7-48eb-b30f-3733137c19cf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:32.501573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 482/943] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Aug 29 23:31:32 2022] GET /identity/v3/services => generated 1497 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:31:32.520331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:32.608623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be7265-680d-4e1b-8e97-3e5fd383f5e9 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:32.668025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff644bc5-d779-44e9-a8b1-20e708d31f1a None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:32.668289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff644bc5-d779-44e9-a8b1-20e708d31f1a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:32.668733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff644bc5-d779-44e9-a8b1-20e708d31f1a None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:32.670792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 483/944] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:31:32.676040 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec2b63-f3d1-4d84-8707-ca6d4813336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:32.677165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec2b63-f3d1-4d84-8707-ca6d4813336a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:32.677660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec2b63-f3d1-4d84-8707-ca6d4813336a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:32.678250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec2b63-f3d1-4d84-8707-ca6d4813336a None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:32.679594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec2b63-f3d1-4d84-8707-ca6d4813336a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:32.680097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766f71b-d988-42b5-af8c-4c731a072125 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:32.680287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec2b63-f3d1-4d84-8707-ca6d4813336a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:32.680422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766f71b-d988-42b5-af8c-4c731a072125 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:32.680727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766f71b-d988-42b5-af8c-4c731a072125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:32.682563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec2b63-f3d1-4d84-8707-ca6d4813336a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:32.709830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8766f71b-d988-42b5-af8c-4c731a072125 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:32.711096 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8766f71b-d988-42b5-af8c-4c731a072125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:32.715397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 462/945] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Aug 29 23:31:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:32.721350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec2b63-f3d1-4d84-8707-ca6d4813336a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:32.824604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 484/946] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:32.894233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec2b63-f3d1-4d84-8707-ca6d4813336a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:32.922468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:32.940586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6fb1f-f7ea-4301-b86e-d6da1c60c50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:32.941248 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6fb1f-f7ea-4301-b86e-d6da1c60c50b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:32.944180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6fb1f-f7ea-4301-b86e-d6da1c60c50b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:32.944180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6fb1f-f7ea-4301-b86e-d6da1c60c50b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:32.955097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6fb1f-f7ea-4301-b86e-d6da1c60c50b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:32.956596 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6fb1f-f7ea-4301-b86e-d6da1c60c50b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-18b6fb1f-f7ea-4301-b86e-d6da1c60c50b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:32.974517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:32.976289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:32.978005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:32.978005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:32.978005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:32.978005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 463/947] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:32.984205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766f71b-d988-42b5-af8c-4c731a072125 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:33.099027 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:33.180971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766f71b-d988-42b5-af8c-4c731a072125 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:33.245560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b5e28-77ca-4513-a66a-8c0f91eaf69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:33.246698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b5e28-77ca-4513-a66a-8c0f91eaf69b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:33.247705 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b5e28-77ca-4513-a66a-8c0f91eaf69b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:33.248180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b5e28-77ca-4513-a66a-8c0f91eaf69b None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:33.249339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b5e28-77ca-4513-a66a-8c0f91eaf69b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:33.249863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b5e28-77ca-4513-a66a-8c0f91eaf69b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:33.252472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b5e28-77ca-4513-a66a-8c0f91eaf69b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:33.274777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 485/948] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:33.281533 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6fb1f-f7ea-4301-b86e-d6da1c60c50b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:33.365342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b5e28-77ca-4513-a66a-8c0f91eaf69b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:33.401135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6fb1f-f7ea-4301-b86e-d6da1c60c50b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:33.443032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc8a1f-a1d7-490c-93f0-561b837304c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:33.443668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc8a1f-a1d7-490c-93f0-561b837304c7 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:33.443946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc8a1f-a1d7-490c-93f0-561b837304c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:33.444205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc8a1f-a1d7-490c-93f0-561b837304c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:33.452188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc8a1f-a1d7-490c-93f0-561b837304c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:33.453657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc8a1f-a1d7-490c-93f0-561b837304c7 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-e9fc8a1f-a1d7-490c-93f0-561b837304c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:33.467508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:33.469666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:33.471578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:33.471578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:33.471578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:33.471578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:33.471578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:33.471578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:33.471578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:33.471578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:33.473544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 464/949] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:33.474727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc8a1f-a1d7-490c-93f0-561b837304c7 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:33.509861 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:33.597690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc8a1f-a1d7-490c-93f0-561b837304c7 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:33.608862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b5e28-77ca-4513-a66a-8c0f91eaf69b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:33.642949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:33.657074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070690-ecd6-45be-8368-39085a1a855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:33.657970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070690-ecd6-45be-8368-39085a1a855f None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:33.658315 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070690-ecd6-45be-8368-39085a1a855f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:33.658538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070690-ecd6-45be-8368-39085a1a855f None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:33.659202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40070690-ecd6-45be-8368-39085a1a855f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:33.660719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40070690-ecd6-45be-8368-39085a1a855f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:33.664753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43582f07-42a4-4e24-8a5e-86c4eb318c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:33.668000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43582f07-42a4-4e24-8a5e-86c4eb318c3d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:33.668000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43582f07-42a4-4e24-8a5e-86c4eb318c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:33.668000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43582f07-42a4-4e24-8a5e-86c4eb318c3d None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:33.669337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43582f07-42a4-4e24-8a5e-86c4eb318c3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:33.670105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43582f07-42a4-4e24-8a5e-86c4eb318c3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:33.674839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43582f07-42a4-4e24-8a5e-86c4eb318c3d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:33.701933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 465/950] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:33.705980 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43582f07-42a4-4e24-8a5e-86c4eb318c3d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:33.716083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.sql.core [None req-40070690-ecd6-45be-8368-39085a1a855f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 23:31:33.716083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 23:31:33.716083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [parameters: {'user_id': 'aadeead1b9ab495d9884ba1abd893cc3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 23:31:33.716083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81852) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-40070690-ecd6-45be-8368-39085a1a855f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application conn.query(q) Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result.read() Aug 29 23:31:33.725823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 23:31:33.727015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application next(self.gen) Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application yield resource Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application next(self.gen) Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise self.value Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application session.commit() Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 23:31:33.728175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.session.flush() Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application compat.raise_( Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 23:31:33.729336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 23:31:33.730536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application conn.query(q) Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result.read() Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 23:31:33.732777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [parameters: {'user_id': 'aadeead1b9ab495d9884ba1abd893cc3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:33.733878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 29 23:31:33.734978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 29 23:31:33.736222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 29 23:31:33.736222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 29 23:31:33.736222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 23:31:33.736222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 23:31:33.736222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 23:31:33.736222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:33.738947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 486/951] 10.222.0.58 () {60 vars in 1131 bytes} [Mon Aug 29 23:31:33 2022] POST /identity/v3/users => generated 160 bytes in 375 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:33.749778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070690-ecd6-45be-8368-39085a1a855f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:33.901908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:33.919965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43582f07-42a4-4e24-8a5e-86c4eb318c3d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:33.974900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070690-ecd6-45be-8368-39085a1a855f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:33.996002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91df1d8-4714-4649-9608-2999fa919e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:33.998856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91df1d8-4714-4649-9608-2999fa919e4e None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:33.999570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91df1d8-4714-4649-9608-2999fa919e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:34.000460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91df1d8-4714-4649-9608-2999fa919e4e None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/ff5b4f2f41604a8286141bce23db35c5/roles/c6a4683645ce4b68a14f2edeee414d92` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:34.005064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91df1d8-4714-4649-9608-2999fa919e4e None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=ff5b4f2f41604a8286141bce23db35c5, role_id=c6a4683645ce4b68a14f2edeee414d92)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:34.008798 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91df1d8-4714-4649-9608-2999fa919e4e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:34.050145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d269a-56f3-4a2a-834f-442e391a34aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:34.051629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d269a-56f3-4a2a-834f-442e391a34aa None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:34.052093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d269a-56f3-4a2a-834f-442e391a34aa None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:34.052534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d269a-56f3-4a2a-834f-442e391a34aa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:34.069400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d269a-56f3-4a2a-834f-442e391a34aa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:34.072781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d269a-56f3-4a2a-834f-442e391a34aa None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-163d269a-56f3-4a2a-834f-442e391a34aa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:34.099033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:34.100090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:34.101262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:34.101262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:34.101262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:34.101262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:34.101262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:34.101262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:34.101262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:34.101262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:34.101262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 23:31:34.101262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:34.101816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 487/952] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:31:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:34.111676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d269a-56f3-4a2a-834f-442e391a34aa None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:34.137545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 466/953] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:31:33 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/ff5b4f2f41604a8286141bce23db35c5/roles/c6a4683645ce4b68a14f2edeee414d92 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:31:34.206370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:34.301945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d269a-56f3-4a2a-834f-442e391a34aa None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:34.343361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b7975-0dc5-4231-bb6d-3ec033d9f9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:34.343980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b7975-0dc5-4231-bb6d-3ec033d9f9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:34.344262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b7975-0dc5-4231-bb6d-3ec033d9f9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:34.344536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b7975-0dc5-4231-bb6d-3ec033d9f9a3 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:34.345260 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b7975-0dc5-4231-bb6d-3ec033d9f9a3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:34.345560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b7975-0dc5-4231-bb6d-3ec033d9f9a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:34.347081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b7975-0dc5-4231-bb6d-3ec033d9f9a3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:34.393258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 488/954] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Aug 29 23:31:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:34.761908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49358bc3-2bda-469f-9651-b2f085430f41 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:34.762194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49358bc3-2bda-469f-9651-b2f085430f41 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:34.763067 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49358bc3-2bda-469f-9651-b2f085430f41 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:34.765161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 467/955] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:31:34.774497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f50a7-5a2b-4352-ad25-51d46c5395e3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:34.774994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f50a7-5a2b-4352-ad25-51d46c5395e3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:34.775485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f50a7-5a2b-4352-ad25-51d46c5395e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:34.806916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-510f50a7-5a2b-4352-ad25-51d46c5395e3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:34.810141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-510f50a7-5a2b-4352-ad25-51d46c5395e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:34.934927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 489/956] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:35.433294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49358bc3-2bda-469f-9651-b2f085430f41 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:35.570411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49358bc3-2bda-469f-9651-b2f085430f41 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:35.613665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d6f33-6981-4a47-b992-1b59b472cf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:35.614194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d6f33-6981-4a47-b992-1b59b472cf42 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:35.614524 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d6f33-6981-4a47-b992-1b59b472cf42 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:35.614772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d6f33-6981-4a47-b992-1b59b472cf42 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:35.615340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d6f33-6981-4a47-b992-1b59b472cf42 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:35.616694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d6f33-6981-4a47-b992-1b59b472cf42 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-966d6f33-6981-4a47-b992-1b59b472cf42 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:35.627240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:35.629221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:35.630965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 468/957] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:35.637373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f50a7-5a2b-4352-ad25-51d46c5395e3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:35.665521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2e42-5e50-4ed1-b924-fbd50f2a1975 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:35.665975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2e42-5e50-4ed1-b924-fbd50f2a1975 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:35.666455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2e42-5e50-4ed1-b924-fbd50f2a1975 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:35.669024 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 469/958] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:35.674994 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e9bb4-3beb-4eff-bf49-65e2f0f717df None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:35.675896 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e9bb4-3beb-4eff-bf49-65e2f0f717df None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:35.675896 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e9bb4-3beb-4eff-bf49-65e2f0f717df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:35.683206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:35.695347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-432e9bb4-3beb-4eff-bf49-65e2f0f717df None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:35.696853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-432e9bb4-3beb-4eff-bf49-65e2f0f717df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:35.711539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:35.786906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f50a7-5a2b-4352-ad25-51d46c5395e3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:35.794506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 470/959] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:35.844404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb671-8ae4-4c74-89b0-a2ea65c18e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:35.845017 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb671-8ae4-4c74-89b0-a2ea65c18e9d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:35.845289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb671-8ae4-4c74-89b0-a2ea65c18e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:35.845560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb671-8ae4-4c74-89b0-a2ea65c18e9d None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:35.846193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfb671-8ae4-4c74-89b0-a2ea65c18e9d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:35.846525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfb671-8ae4-4c74-89b0-a2ea65c18e9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:35.847897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfb671-8ae4-4c74-89b0-a2ea65c18e9d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:35.853631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 490/960] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:31:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:35.861025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e9bb4-3beb-4eff-bf49-65e2f0f717df None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:35.991437 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e9bb4-3beb-4eff-bf49-65e2f0f717df None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:36.033944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8b130-1113-4f1f-b11f-677351b81ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:36.034605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b130-1113-4f1f-b11f-677351b81ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:36.034888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b130-1113-4f1f-b11f-677351b81ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:36.035157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b130-1113-4f1f-b11f-677351b81ef4 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:36.035895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8b130-1113-4f1f-b11f-677351b81ef4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:36.036200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8b130-1113-4f1f-b11f-677351b81ef4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:36.037621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8b130-1113-4f1f-b11f-677351b81ef4 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:36.050468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 471/961] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:31:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:36.323390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb671-8ae4-4c74-89b0-a2ea65c18e9d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:36.459485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:36.477382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb671-8ae4-4c74-89b0-a2ea65c18e9d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:36.531378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160fd88-6907-49af-9387-42d92b36ce87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:36.532332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160fd88-6907-49af-9387-42d92b36ce87 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:36.532611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160fd88-6907-49af-9387-42d92b36ce87 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:36.532878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160fd88-6907-49af-9387-42d92b36ce87 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:36.533460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160fd88-6907-49af-9387-42d92b36ce87 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:36.534888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160fd88-6907-49af-9387-42d92b36ce87 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1160fd88-6907-49af-9387-42d92b36ce87 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:36.545641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:36.547597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:36.550216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:36.553188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 491/962] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Aug 29 23:31:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:36.555240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8b130-1113-4f1f-b11f-677351b81ef4 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:36.572084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:36.718159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8b130-1113-4f1f-b11f-677351b81ef4 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:36.782992 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d991c9c-2c05-410b-89e4-b4c54bcb643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:36.784046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d991c9c-2c05-410b-89e4-b4c54bcb643c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:36.784528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d991c9c-2c05-410b-89e4-b4c54bcb643c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:36.784997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d991c9c-2c05-410b-89e4-b4c54bcb643c None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:36.786120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d991c9c-2c05-410b-89e4-b4c54bcb643c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:36.788319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d991c9c-2c05-410b-89e4-b4c54bcb643c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:36.789175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d991c9c-2c05-410b-89e4-b4c54bcb643c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:36.798593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 472/963] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Aug 29 23:31:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:36.805329 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160fd88-6907-49af-9387-42d92b36ce87 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:36.995200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160fd88-6907-49af-9387-42d92b36ce87 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:37.063590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e7e33-dcf8-4c0b-bbce-8b6bf363572f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:37.064606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7e33-dcf8-4c0b-bbce-8b6bf363572f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:37.065110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7e33-dcf8-4c0b-bbce-8b6bf363572f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:37.065598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7e33-dcf8-4c0b-bbce-8b6bf363572f None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:37.066794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e7e33-dcf8-4c0b-bbce-8b6bf363572f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:37.067321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e7e33-dcf8-4c0b-bbce-8b6bf363572f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:37.069896 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e7e33-dcf8-4c0b-bbce-8b6bf363572f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:37.080952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 492/964] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Aug 29 23:31:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:37.089629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d991c9c-2c05-410b-89e4-b4c54bcb643c None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:37.104126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf1bec-81b5-4a70-8e1f-a65a7c369c25 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:37.104566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf1bec-81b5-4a70-8e1f-a65a7c369c25 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:37.105005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf1bec-81b5-4a70-8e1f-a65a7c369c25 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:37.108070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 493/965] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:37.116944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41e8ee-4e54-4a1a-b05b-ec96d2d59e86 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:37.117262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41e8ee-4e54-4a1a-b05b-ec96d2d59e86 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:37.117642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41e8ee-4e54-4a1a-b05b-ec96d2d59e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:37.146623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4b41e8ee-4e54-4a1a-b05b-ec96d2d59e86 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:37.148456 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4b41e8ee-4e54-4a1a-b05b-ec96d2d59e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:37.172573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:37.228353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:37.269566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 494/966] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:37.270092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d991c9c-2c05-410b-89e4-b4c54bcb643c None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:37.311407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56be69c-0ff2-403e-a123-37d28f078efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:37.312027 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56be69c-0ff2-403e-a123-37d28f078efe None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:37.312356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56be69c-0ff2-403e-a123-37d28f078efe None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:37.312635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56be69c-0ff2-403e-a123-37d28f078efe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:37.313335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56be69c-0ff2-403e-a123-37d28f078efe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:37.314752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56be69c-0ff2-403e-a123-37d28f078efe None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:37.421066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 473/967] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:31:37 2022] POST /identity/v3/endpoints => generated 349 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:31:37.654598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41e8ee-4e54-4a1a-b05b-ec96d2d59e86 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:37.771372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41e8ee-4e54-4a1a-b05b-ec96d2d59e86 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:37.820018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19e4f6-3969-4967-9c59-664c9570a798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:37.820654 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19e4f6-3969-4967-9c59-664c9570a798 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:37.820917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19e4f6-3969-4967-9c59-664c9570a798 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:37.821185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19e4f6-3969-4967-9c59-664c9570a798 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:37.838301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19e4f6-3969-4967-9c59-664c9570a798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-5c19e4f6-3969-4967-9c59-664c9570a798 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:37.848347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:37.849365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:31:37.850240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:37.851766 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 495/968] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Aug 29 23:31:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:37.859744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56be69c-0ff2-403e-a123-37d28f078efe None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:38.011028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56be69c-0ff2-403e-a123-37d28f078efe None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:38.046204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:38.053783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebf7ff-9d61-4c6c-8e4b-795ec1043ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:38.054397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebf7ff-9d61-4c6c-8e4b-795ec1043ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:38.054708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebf7ff-9d61-4c6c-8e4b-795ec1043ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:38.054979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebf7ff-9d61-4c6c-8e4b-795ec1043ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:38.055664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebf7ff-9d61-4c6c-8e4b-795ec1043ad6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:38.055966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebf7ff-9d61-4c6c-8e4b-795ec1043ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:38.057426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebf7ff-9d61-4c6c-8e4b-795ec1043ad6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:38.067301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 474/969] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Aug 29 23:31:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:38.072960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19e4f6-3969-4967-9c59-664c9570a798 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:38.078459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:38.202946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19e4f6-3969-4967-9c59-664c9570a798 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:38.253286 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbcb0b-dc97-47bf-88a3-25fff110b10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:38.253925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbcb0b-dc97-47bf-88a3-25fff110b10e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:38.254201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbcb0b-dc97-47bf-88a3-25fff110b10e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:38.254513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbcb0b-dc97-47bf-88a3-25fff110b10e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:38.267341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbcb0b-dc97-47bf-88a3-25fff110b10e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:38.268769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbcb0b-dc97-47bf-88a3-25fff110b10e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-26cbcb0b-dc97-47bf-88a3-25fff110b10e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:38.285594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:38.287401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:38.289115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:38.289115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:38.289115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:38.290339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 496/970] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:38.294305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebf7ff-9d61-4c6c-8e4b-795ec1043ad6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:38.440592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebf7ff-9d61-4c6c-8e4b-795ec1043ad6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:38.503145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7a2b0-0131-46c7-9158-fc1a22cd69de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:38.504191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7a2b0-0131-46c7-9158-fc1a22cd69de None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:38.504644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7a2b0-0131-46c7-9158-fc1a22cd69de None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:38.504991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7a2b0-0131-46c7-9158-fc1a22cd69de None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:38.506002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7a2b0-0131-46c7-9158-fc1a22cd69de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:38.506526 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7a2b0-0131-46c7-9158-fc1a22cd69de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:38.509065 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7a2b0-0131-46c7-9158-fc1a22cd69de None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:38.517000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726e629-282d-482d-89e0-811bc89cff25 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:38.517443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726e629-282d-482d-89e0-811bc89cff25 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:38.517738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726e629-282d-482d-89e0-811bc89cff25 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:38.519454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 497/971] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:38.524920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e3193-3bbc-4780-bb49-427b7e9360b1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:38.525245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e3193-3bbc-4780-bb49-427b7e9360b1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:38.526009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e3193-3bbc-4780-bb49-427b7e9360b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:38.531685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 475/972] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:38.539657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7a2b0-0131-46c7-9158-fc1a22cd69de None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:38.546098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-333e3193-3bbc-4780-bb49-427b7e9360b1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:38.547360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-333e3193-3bbc-4780-bb49-427b7e9360b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:38.607182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:38.611972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:38.638754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 498/973] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:38.680304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7a2b0-0131-46c7-9158-fc1a22cd69de None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:38.727117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a61632-2722-42c3-9943-005e4bce214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:38.727764 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a61632-2722-42c3-9943-005e4bce214f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:38.728051 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a61632-2722-42c3-9943-005e4bce214f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:38.728325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a61632-2722-42c3-9943-005e4bce214f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:38.737484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a61632-2722-42c3-9943-005e4bce214f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:38.738999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a61632-2722-42c3-9943-005e4bce214f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-13a61632-2722-42c3-9943-005e4bce214f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:38.751653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:38.759331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:38.761291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:38.761291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:38.761291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:38.761291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:38.761291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:38.761291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:38.761291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:38.761291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:38.761291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 23:31:38.761291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:38.761291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 476/974] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Aug 29 23:31:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:38.767969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e3193-3bbc-4780-bb49-427b7e9360b1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:38.895145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e3193-3bbc-4780-bb49-427b7e9360b1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:38.916597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a61632-2722-42c3-9943-005e4bce214f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:38.938606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cf956-9a41-4ec1-8ba1-7819b7739af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:38.939334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cf956-9a41-4ec1-8ba1-7819b7739af1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:38.939721 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cf956-9a41-4ec1-8ba1-7819b7739af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:38.940003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cf956-9a41-4ec1-8ba1-7819b7739af1 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:38.940778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1cf956-9a41-4ec1-8ba1-7819b7739af1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:38.941216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1cf956-9a41-4ec1-8ba1-7819b7739af1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:38.943429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1cf956-9a41-4ec1-8ba1-7819b7739af1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:38.978461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 499/975] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Aug 29 23:31:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:38.983071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cf956-9a41-4ec1-8ba1-7819b7739af1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:39.037242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:39.041746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a61632-2722-42c3-9943-005e4bce214f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:39.057706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:39.081855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f35d32-0e04-489b-bd60-359760b08c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:39.082551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35d32-0e04-489b-bd60-359760b08c3a None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:39.082828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35d32-0e04-489b-bd60-359760b08c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:39.083101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35d32-0e04-489b-bd60-359760b08c3a None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:39.083781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f35d32-0e04-489b-bd60-359760b08c3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:39.085149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f35d32-0e04-489b-bd60-359760b08c3a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:39.099961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cf956-9a41-4ec1-8ba1-7819b7739af1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:39.141436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439dab9-fbbf-4ec4-881a-95de557770fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:39.142046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439dab9-fbbf-4ec4-881a-95de557770fd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:39.142326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439dab9-fbbf-4ec4-881a-95de557770fd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:39.142644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439dab9-fbbf-4ec4-881a-95de557770fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:39.219378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439dab9-fbbf-4ec4-881a-95de557770fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:39.220857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439dab9-fbbf-4ec4-881a-95de557770fd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:39.222466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 477/976] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:31:38 2022] POST /identity/v3/services => generated 242 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:31:39.302751 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:39.414563 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 500/977] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:31:38 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:31:39.421373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f35d32-0e04-489b-bd60-359760b08c3a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:39.572893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f35d32-0e04-489b-bd60-359760b08c3a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:39.614734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b4862-46eb-4e28-a09c-63814cf536e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:39.615386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b4862-46eb-4e28-a09c-63814cf536e8 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:39.615665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b4862-46eb-4e28-a09c-63814cf536e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:39.615940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b4862-46eb-4e28-a09c-63814cf536e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:39.629425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b4862-46eb-4e28-a09c-63814cf536e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:39.630939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b4862-46eb-4e28-a09c-63814cf536e8 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-892b4862-46eb-4e28-a09c-63814cf536e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:39.646993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:39.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:39.649158 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:39.649158 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:39.649158 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:39.649158 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 478/978] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:39.655455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439dab9-fbbf-4ec4-881a-95de557770fd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:39.748476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58e0f1-254a-431a-a594-7c80a5644e3e None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:39.748954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58e0f1-254a-431a-a594-7c80a5644e3e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:39.749244 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58e0f1-254a-431a-a594-7c80a5644e3e None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:39.750920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 479/979] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:31:39.758822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86437773-a208-47f8-95b8-640035314d0d None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:39.759541 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86437773-a208-47f8-95b8-640035314d0d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:39.760036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86437773-a208-47f8-95b8-640035314d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:39.776499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439dab9-fbbf-4ec4-881a-95de557770fd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:39.786514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-86437773-a208-47f8-95b8-640035314d0d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:39.788287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-86437773-a208-47f8-95b8-640035314d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:39.807157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:39.818078 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0dab7e-3ee4-4941-bc1b-e1a9cee35499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:39.829424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dab7e-3ee4-4941-bc1b-e1a9cee35499 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:39.829424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dab7e-3ee4-4941-bc1b-e1a9cee35499 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:39.829424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dab7e-3ee4-4941-bc1b-e1a9cee35499 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:39.829424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0dab7e-3ee4-4941-bc1b-e1a9cee35499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:39.829424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0dab7e-3ee4-4941-bc1b-e1a9cee35499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:39.829424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0dab7e-3ee4-4941-bc1b-e1a9cee35499 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:39.842756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 501/980] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:39.845381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0dab7e-3ee4-4941-bc1b-e1a9cee35499 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:39.851230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:39.879057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 480/981] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:40.009557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0dab7e-3ee4-4941-bc1b-e1a9cee35499 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:40.073358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e0fda-c829-41ce-b719-7472994cfa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:40.075471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e0fda-c829-41ce-b719-7472994cfa7d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:40.076207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e0fda-c829-41ce-b719-7472994cfa7d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:40.076881 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e0fda-c829-41ce-b719-7472994cfa7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:40.089330 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e0fda-c829-41ce-b719-7472994cfa7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:40.092185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e0fda-c829-41ce-b719-7472994cfa7d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-7b5e0fda-c829-41ce-b719-7472994cfa7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:40.105242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:40.115431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:40.117604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:40.117604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:40.117604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:40.117604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:40.117604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:40.117604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:40.117604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:40.117604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:40.117604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 502/982] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:40.121962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86437773-a208-47f8-95b8-640035314d0d None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:40.158626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e0fda-c829-41ce-b719-7472994cfa7d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:40.249242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:40.268351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86437773-a208-47f8-95b8-640035314d0d None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:40.301044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e0fda-c829-41ce-b719-7472994cfa7d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:40.315341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9ef18-b5b8-4e7f-af3d-e0776a2c03d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:40.316434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9ef18-b5b8-4e7f-af3d-e0776a2c03d1 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:40.317283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9ef18-b5b8-4e7f-af3d-e0776a2c03d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:40.318035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9ef18-b5b8-4e7f-af3d-e0776a2c03d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:40.318991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9ef18-b5b8-4e7f-af3d-e0776a2c03d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:40.319632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9ef18-b5b8-4e7f-af3d-e0776a2c03d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:40.321624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9ef18-b5b8-4e7f-af3d-e0776a2c03d1 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:40.338530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bd5a9-2a1a-4eb1-875b-66e1078a6d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:40.339181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bd5a9-2a1a-4eb1-875b-66e1078a6d07 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:40.339526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bd5a9-2a1a-4eb1-875b-66e1078a6d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:40.339738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bd5a9-2a1a-4eb1-875b-66e1078a6d07 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:40.340116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 481/983] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:40.345863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bd5a9-2a1a-4eb1-875b-66e1078a6d07 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:40.345863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bd5a9-2a1a-4eb1-875b-66e1078a6d07 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:40.347787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9ef18-b5b8-4e7f-af3d-e0776a2c03d1 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-974bd5a9-2a1a-4eb1-875b-66e1078a6d07 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:40.359162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:40.360327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:40.361207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 503/984] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Aug 29 23:31:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:40.366635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bd5a9-2a1a-4eb1-875b-66e1078a6d07 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:40.368634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:40.502251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bd5a9-2a1a-4eb1-875b-66e1078a6d07 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:40.503278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9ef18-b5b8-4e7f-af3d-e0776a2c03d1 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:40.538928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:40.555792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15565e8-5917-4f33-b3c0-d8440baff2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:40.556931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15565e8-5917-4f33-b3c0-d8440baff2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:40.557423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15565e8-5917-4f33-b3c0-d8440baff2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:40.557873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15565e8-5917-4f33-b3c0-d8440baff2f8 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:40.564518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15565e8-5917-4f33-b3c0-d8440baff2f8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:40.564518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15565e8-5917-4f33-b3c0-d8440baff2f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:40.564518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15565e8-5917-4f33-b3c0-d8440baff2f8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:40.573480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de47b9-6fe8-4fb6-b1ce-186a71a7430f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:40.575078 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de47b9-6fe8-4fb6-b1ce-186a71a7430f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:40.575576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de47b9-6fe8-4fb6-b1ce-186a71a7430f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:40.576049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de47b9-6fe8-4fb6-b1ce-186a71a7430f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:40.577450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de47b9-6fe8-4fb6-b1ce-186a71a7430f None admin] RBAC: Adding query filter params (role.id=c6a4683645ce4b68a14f2edeee414d92, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=ff5b4f2f41604a8286141bce23db35c5) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:40.577987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de47b9-6fe8-4fb6-b1ce-186a71a7430f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:40.578988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 504/985] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Aug 29 23:31:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:40.580776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de47b9-6fe8-4fb6-b1ce-186a71a7430f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:40.583615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15565e8-5917-4f33-b3c0-d8440baff2f8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:40.594947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 482/986] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:31:40 2022] GET /identity/v3/role_assignments?user.id=ff5b4f2f41604a8286141bce23db35c5&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=c6a4683645ce4b68a14f2edeee414d92 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:40.775398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15565e8-5917-4f33-b3c0-d8440baff2f8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:40.832070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f1dc-007b-4fde-83e0-bccb3cfdc298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:40.832701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f1dc-007b-4fde-83e0-bccb3cfdc298 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:40.832985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f1dc-007b-4fde-83e0-bccb3cfdc298 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:40.833246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f1dc-007b-4fde-83e0-bccb3cfdc298 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:40.834022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479f1dc-007b-4fde-83e0-bccb3cfdc298 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:40.834325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479f1dc-007b-4fde-83e0-bccb3cfdc298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:40.835837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479f1dc-007b-4fde-83e0-bccb3cfdc298 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:40.842363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 505/987] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Aug 29 23:31:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:40.848860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de47b9-6fe8-4fb6-b1ce-186a71a7430f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:40.979674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:41.026079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de47b9-6fe8-4fb6-b1ce-186a71a7430f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:41.093125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad25f0-3b82-4ba5-8668-21da291b633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:41.094179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad25f0-3b82-4ba5-8668-21da291b633a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:41.094710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad25f0-3b82-4ba5-8668-21da291b633a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:41.095201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad25f0-3b82-4ba5-8668-21da291b633a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:41.096549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad25f0-3b82-4ba5-8668-21da291b633a None admin] RBAC: Adding query filter params (interface=admin, service_id=e2eee657072340efa0bff20c8e96634f, region_id=RegionOne) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:41.097103 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad25f0-3b82-4ba5-8668-21da291b633a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:41.099919 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad25f0-3b82-4ba5-8668-21da291b633a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:41.115893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 483/988] 10.222.0.58 () {56 vars in 1249 bytes} [Mon Aug 29 23:31:40 2022] GET /identity/v3/endpoints?service_id=e2eee657072340efa0bff20c8e96634f&interface=admin®ion_id=RegionOne => generated 193 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:41.117269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f1dc-007b-4fde-83e0-bccb3cfdc298 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:41.186218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:41.264712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f1dc-007b-4fde-83e0-bccb3cfdc298 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:41.333712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080f622-4782-4ccd-b386-81b4801af885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:41.335557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080f622-4782-4ccd-b386-81b4801af885 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:41.336348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080f622-4782-4ccd-b386-81b4801af885 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:41.337125 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080f622-4782-4ccd-b386-81b4801af885 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:41.338447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080f622-4782-4ccd-b386-81b4801af885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:41.341677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080f622-4782-4ccd-b386-81b4801af885 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:41.358553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 506/989] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Aug 29 23:31:41 2022] GET /identity/v3/services => generated 1727 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:31:41.599953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5281a-d0ff-4b5c-8ec6-b36685e32d08 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:41.600709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5281a-d0ff-4b5c-8ec6-b36685e32d08 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:41.600709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5281a-d0ff-4b5c-8ec6-b36685e32d08 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:41.601815 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 484/990] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:41.610970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c2ea5-9f60-45cb-9be8-b39ee2bf4da4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:41.611195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c2ea5-9f60-45cb-9be8-b39ee2bf4da4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:41.611457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c2ea5-9f60-45cb-9be8-b39ee2bf4da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:41.643813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-823c2ea5-9f60-45cb-9be8-b39ee2bf4da4 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:41.645908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-823c2ea5-9f60-45cb-9be8-b39ee2bf4da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:41.778567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 507/991] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:42.297451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5281a-d0ff-4b5c-8ec6-b36685e32d08 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:42.472096 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5281a-d0ff-4b5c-8ec6-b36685e32d08 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:42.515069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca54e5-940a-425b-8724-0f6a6fcacc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:42.515877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca54e5-940a-425b-8724-0f6a6fcacc8d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:42.516282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca54e5-940a-425b-8724-0f6a6fcacc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:42.516672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca54e5-940a-425b-8724-0f6a6fcacc8d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:42.517426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca54e5-940a-425b-8724-0f6a6fcacc8d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:42.518971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca54e5-940a-425b-8724-0f6a6fcacc8d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-eaca54e5-940a-425b-8724-0f6a6fcacc8d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:42.529044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:42.531157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:42.532115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 485/992] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:42.540921 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c2ea5-9f60-45cb-9be8-b39ee2bf4da4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:42.647200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:42.744992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c2ea5-9f60-45cb-9be8-b39ee2bf4da4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:42.815665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20634b5-debe-4cb0-99ef-1173bdc641cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:42.816630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20634b5-debe-4cb0-99ef-1173bdc641cc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:42.817111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20634b5-debe-4cb0-99ef-1173bdc641cc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:42.817594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20634b5-debe-4cb0-99ef-1173bdc641cc None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:42.818755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20634b5-debe-4cb0-99ef-1173bdc641cc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:42.819277 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20634b5-debe-4cb0-99ef-1173bdc641cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:42.821745 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20634b5-debe-4cb0-99ef-1173bdc641cc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:42.838642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 508/993] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:31:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:42.844128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca54e5-940a-425b-8724-0f6a6fcacc8d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:42.849863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:42.975512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fa67-d769-44db-bc31-076d33c2aad6 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:42.975840 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fa67-d769-44db-bc31-076d33c2aad6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:42.976305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fa67-d769-44db-bc31-076d33c2aad6 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:42.978010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 509/994] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:31:42.984980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017aeaa-cc10-481c-b0ba-fdcc0714b0fc None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:42.985427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017aeaa-cc10-481c-b0ba-fdcc0714b0fc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:42.985845 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017aeaa-cc10-481c-b0ba-fdcc0714b0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:43.007052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b017aeaa-cc10-481c-b0ba-fdcc0714b0fc None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:43.008703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b017aeaa-cc10-481c-b0ba-fdcc0714b0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:43.042502 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca54e5-940a-425b-8724-0f6a6fcacc8d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:43.100750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 510/995] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:43.109586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ae717-2984-442a-899f-e5054945ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:43.110657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae717-2984-442a-899f-e5054945ea83 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:43.111139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae717-2984-442a-899f-e5054945ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:43.111633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae717-2984-442a-899f-e5054945ea83 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:43.112773 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ae717-2984-442a-899f-e5054945ea83 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:43.113311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ae717-2984-442a-899f-e5054945ea83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:43.115902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ae717-2984-442a-899f-e5054945ea83 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:43.133727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 486/996] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:31:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:43.135418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017aeaa-cc10-481c-b0ba-fdcc0714b0fc None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:43.218012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:43.316899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017aeaa-cc10-481c-b0ba-fdcc0714b0fc None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:43.385317 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350c761-9a07-486a-a29c-79cd236d90ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:43.386193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c761-9a07-486a-a29c-79cd236d90ca None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:43.387643 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c761-9a07-486a-a29c-79cd236d90ca None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:43.388117 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c761-9a07-486a-a29c-79cd236d90ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:43.389202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350c761-9a07-486a-a29c-79cd236d90ca None admin] RBAC: Adding query filter params (interface=public, service_id=2b745be463f84c9da0f4bad85de73019, region_id=RegionOne) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:43.389922 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350c761-9a07-486a-a29c-79cd236d90ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:43.392482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350c761-9a07-486a-a29c-79cd236d90ca None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:43.406388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 511/997] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Aug 29 23:31:43 2022] GET /identity/v3/endpoints?service_id=2b745be463f84c9da0f4bad85de73019&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:43.407847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ae717-2984-442a-899f-e5054945ea83 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:43.461904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5291f7-e348-490c-836f-abfd24d68a2c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:43.469884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5291f7-e348-490c-836f-abfd24d68a2c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:43.469884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5291f7-e348-490c-836f-abfd24d68a2c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:43.469884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 512/998] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:43.472823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07339e8d-69f3-4757-9e08-44c1ced16b36 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:43.473305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07339e8d-69f3-4757-9e08-44c1ced16b36 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:43.473586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07339e8d-69f3-4757-9e08-44c1ced16b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:43.493683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-07339e8d-69f3-4757-9e08-44c1ced16b36 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:43.494954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-07339e8d-69f3-4757-9e08-44c1ced16b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:43.509288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:43.568203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ae717-2984-442a-899f-e5054945ea83 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:43.601258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 513/999] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:43.602277 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07339e8d-69f3-4757-9e08-44c1ced16b36 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:43.627461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6765b-0fee-4bda-a2cb-90e3ab871346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:43.628663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6765b-0fee-4bda-a2cb-90e3ab871346 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:43.629288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6765b-0fee-4bda-a2cb-90e3ab871346 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:43.629688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6765b-0fee-4bda-a2cb-90e3ab871346 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:43.630817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6765b-0fee-4bda-a2cb-90e3ab871346 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:43.633118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6765b-0fee-4bda-a2cb-90e3ab871346 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:43.658462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 487/1000] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Aug 29 23:31:43 2022] GET /identity/v3/services => generated 1727 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:31:43.680171 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:43.758564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07339e8d-69f3-4757-9e08-44c1ced16b36 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:43.801825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e693b7c-4208-497d-9e43-848d5a3a571e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:43.802508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e693b7c-4208-497d-9e43-848d5a3a571e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:43.802793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e693b7c-4208-497d-9e43-848d5a3a571e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:43.803052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e693b7c-4208-497d-9e43-848d5a3a571e None admin] PATH_INFO: `/v3/services/image` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:43.803648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e693b7c-4208-497d-9e43-848d5a3a571e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:43.805066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e693b7c-4208-497d-9e43-848d5a3a571e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-4e693b7c-4208-497d-9e43-848d5a3a571e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:43.815925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:43.817826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:43.819505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 514/1001] 10.222.0.58 () {56 vars in 1105 bytes} [Mon Aug 29 23:31:43 2022] GET /identity/v3/services/image => generated 86 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:43.828087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6765b-0fee-4bda-a2cb-90e3ab871346 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:43.890642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e693b7c-4208-497d-9e43-848d5a3a571e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:44.014765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6765b-0fee-4bda-a2cb-90e3ab871346 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:44.024592 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e693b7c-4208-497d-9e43-848d5a3a571e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:44.027888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:44.048617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:44.055518 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab23bd-efa4-48c2-9a22-5f07a92ba434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:44.056134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab23bd-efa4-48c2-9a22-5f07a92ba434 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:44.056426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab23bd-efa4-48c2-9a22-5f07a92ba434 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:44.056691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab23bd-efa4-48c2-9a22-5f07a92ba434 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:44.057336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab23bd-efa4-48c2-9a22-5f07a92ba434 None admin] RBAC: Adding query filter params (name=image) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:44.057638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab23bd-efa4-48c2-9a22-5f07a92ba434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:44.059137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab23bd-efa4-48c2-9a22-5f07a92ba434 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:44.065137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 488/1002] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:44.073266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab23bd-efa4-48c2-9a22-5f07a92ba434 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:44.086316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9ec3a-eb06-43a8-bb7b-259ef921dd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:44.087183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9ec3a-eb06-43a8-bb7b-259ef921dd4a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:44.087505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9ec3a-eb06-43a8-bb7b-259ef921dd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:44.087786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9ec3a-eb06-43a8-bb7b-259ef921dd4a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:44.088493 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9ec3a-eb06-43a8-bb7b-259ef921dd4a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:44.090056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9ec3a-eb06-43a8-bb7b-259ef921dd4a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-86f9ec3a-eb06-43a8-bb7b-259ef921dd4a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:44.100175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:44.101512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:44.102645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 515/1003] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Aug 29 23:31:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:44.108574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9ec3a-eb06-43a8-bb7b-259ef921dd4a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:44.235199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab23bd-efa4-48c2-9a22-5f07a92ba434 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:44.289168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bc14a-8e65-415a-b40a-91909382a4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:44.289652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9ec3a-eb06-43a8-bb7b-259ef921dd4a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:44.290343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bc14a-8e65-415a-b40a-91909382a4dc None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:44.290815 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bc14a-8e65-415a-b40a-91909382a4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:44.291303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bc14a-8e65-415a-b40a-91909382a4dc None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:44.292305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bc14a-8e65-415a-b40a-91909382a4dc None admin] RBAC: Adding query filter params (type=image) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:44.292989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bc14a-8e65-415a-b40a-91909382a4dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:44.294967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bc14a-8e65-415a-b40a-91909382a4dc None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:44.301510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 489/1004] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:44 2022] GET /identity/v3/services?type=image => generated 122 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:44.342667 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b398f94-85e2-4e75-b1a0-5cf2e73c7cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:44.343468 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b398f94-85e2-4e75-b1a0-5cf2e73c7cbd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:44.343884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b398f94-85e2-4e75-b1a0-5cf2e73c7cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:44.344300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b398f94-85e2-4e75-b1a0-5cf2e73c7cbd None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:44.345101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b398f94-85e2-4e75-b1a0-5cf2e73c7cbd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:44.345548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b398f94-85e2-4e75-b1a0-5cf2e73c7cbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:44.350527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b398f94-85e2-4e75-b1a0-5cf2e73c7cbd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:44.358907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 516/1005] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Aug 29 23:31:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:44.365029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bc14a-8e65-415a-b40a-91909382a4dc None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:44.467436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:44.526786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bc14a-8e65-415a-b40a-91909382a4dc None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:44.579493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d32fb3-fd1d-4e7a-b5e7-7b33f4534223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:44.580113 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d32fb3-fd1d-4e7a-b5e7-7b33f4534223 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:44.580546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d32fb3-fd1d-4e7a-b5e7-7b33f4534223 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:44.580962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d32fb3-fd1d-4e7a-b5e7-7b33f4534223 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:44.581929 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d32fb3-fd1d-4e7a-b5e7-7b33f4534223 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:44.582385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d32fb3-fd1d-4e7a-b5e7-7b33f4534223 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:44.584642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d32fb3-fd1d-4e7a-b5e7-7b33f4534223 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:44.594487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 490/1006] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Aug 29 23:31:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:44.602178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b398f94-85e2-4e75-b1a0-5cf2e73c7cbd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:44.664796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:44.798086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b398f94-85e2-4e75-b1a0-5cf2e73c7cbd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:44.865059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad271b-9947-454e-bb58-287aa08bb9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:44.865993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad271b-9947-454e-bb58-287aa08bb9ba None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:44.866507 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad271b-9947-454e-bb58-287aa08bb9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:44.866963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad271b-9947-454e-bb58-287aa08bb9ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:44.868207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad271b-9947-454e-bb58-287aa08bb9ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:44.870804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad271b-9947-454e-bb58-287aa08bb9ba None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:44.964577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 517/1007] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:31:44 2022] POST /identity/v3/endpoints => generated 325 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:31:45.786000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639e915-56d1-4668-a02b-87ef09628ee4 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:45.786806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639e915-56d1-4668-a02b-87ef09628ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:45.786806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639e915-56d1-4668-a02b-87ef09628ee4 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:45.788053 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 491/1008] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:45.793992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126d26e-a1af-4e15-b696-4072b298c045 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:45.794809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126d26e-a1af-4e15-b696-4072b298c045 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:45.794809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126d26e-a1af-4e15-b696-4072b298c045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:45.814830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e126d26e-a1af-4e15-b696-4072b298c045 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:45.816039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e126d26e-a1af-4e15-b696-4072b298c045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:45.903226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 518/1009] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:45 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:46.177626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639e915-56d1-4668-a02b-87ef09628ee4 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:46.319529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639e915-56d1-4668-a02b-87ef09628ee4 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:46.359288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:46.375052 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093338c-9e71-4697-bbf6-ad317b92a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:46.376826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093338c-9e71-4697-bbf6-ad317b92a619 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:46.376826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093338c-9e71-4697-bbf6-ad317b92a619 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:46.376826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093338c-9e71-4697-bbf6-ad317b92a619 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:46.377429 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093338c-9e71-4697-bbf6-ad317b92a619 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:46.379119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093338c-9e71-4697-bbf6-ad317b92a619 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:46.383875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f901af-7fc8-4e6f-a06e-a90b13a0b4da None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:46.384168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f901af-7fc8-4e6f-a06e-a90b13a0b4da None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:46.384385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f901af-7fc8-4e6f-a06e-a90b13a0b4da None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:46.386071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 519/1010] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-8093338c-9e71-4697-bbf6-ad317b92a619 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:46.391754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:46.394139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 23:31:46.395851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:46.399637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b928c8a-5be8-48a0-9ce7-0184c3a1a518 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:46.399637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b928c8a-5be8-48a0-9ce7-0184c3a1a518 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:46.399637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b928c8a-5be8-48a0-9ce7-0184c3a1a518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:46.403908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 492/1011] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:46.405587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093338c-9e71-4697-bbf6-ad317b92a619 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:46.415132 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7b928c8a-5be8-48a0-9ce7-0184c3a1a518 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:46.416825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7b928c8a-5be8-48a0-9ce7-0184c3a1a518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:46.431785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:46.518483 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 520/1012] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:46.555287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093338c-9e71-4697-bbf6-ad317b92a619 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:46.596291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada18ba-ec19-47d4-adc1-f2514333668d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:46.596903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada18ba-ec19-47d4-adc1-f2514333668d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:46.597180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada18ba-ec19-47d4-adc1-f2514333668d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:46.597467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada18ba-ec19-47d4-adc1-f2514333668d None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:46.598090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada18ba-ec19-47d4-adc1-f2514333668d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:46.598391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada18ba-ec19-47d4-adc1-f2514333668d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:46.599811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada18ba-ec19-47d4-adc1-f2514333668d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:46.612872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 493/1013] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:31:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:46.614205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b928c8a-5be8-48a0-9ce7-0184c3a1a518 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:46.790763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada18ba-ec19-47d4-adc1-f2514333668d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:46.792324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b928c8a-5be8-48a0-9ce7-0184c3a1a518 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:46.856288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbd840-2073-4f33-a6fb-e40f94415975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:46.857313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbd840-2073-4f33-a6fb-e40f94415975 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:46.857507 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbd840-2073-4f33-a6fb-e40f94415975 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:46.857785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbd840-2073-4f33-a6fb-e40f94415975 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:46.858634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbd840-2073-4f33-a6fb-e40f94415975 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:46.859214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbd840-2073-4f33-a6fb-e40f94415975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:46.861265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbd840-2073-4f33-a6fb-e40f94415975 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:46.865866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:46.871500 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 521/1014] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Aug 29 23:31:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:46.880330 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbd840-2073-4f33-a6fb-e40f94415975 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:46.900110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:46.966636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada18ba-ec19-47d4-adc1-f2514333668d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:47.010455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbd840-2073-4f33-a6fb-e40f94415975 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:47.032922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b03e42-d668-4cad-8f35-c6f55c91e4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:47.034196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b03e42-d668-4cad-8f35-c6f55c91e4ff None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:47.034997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b03e42-d668-4cad-8f35-c6f55c91e4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:47.035660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b03e42-d668-4cad-8f35-c6f55c91e4ff None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:47.036897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b03e42-d668-4cad-8f35-c6f55c91e4ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:47.039592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b03e42-d668-4cad-8f35-c6f55c91e4ff None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:47.061473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc348455-de63-4eab-8e2f-e53a429f8a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:47.061473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc348455-de63-4eab-8e2f-e53a429f8a9f None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:47.061473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc348455-de63-4eab-8e2f-e53a429f8a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:47.061473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc348455-de63-4eab-8e2f-e53a429f8a9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:47.062062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc348455-de63-4eab-8e2f-e53a429f8a9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:47.062841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc348455-de63-4eab-8e2f-e53a429f8a9f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:47.115854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 494/1015] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:31:46 2022] POST /identity/v3/services => generated 242 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:31:47.117197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02f97-1cde-4aed-b94a-7bd6d36fa202 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:47.117485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02f97-1cde-4aed-b94a-7bd6d36fa202 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:47.117759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02f97-1cde-4aed-b94a-7bd6d36fa202 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:47.119310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 495/1016] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:47.126248 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f93868-6af7-4c30-8a1f-fabae854c4a3 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:47.126572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f93868-6af7-4c30-8a1f-fabae854c4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:47.126851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f93868-6af7-4c30-8a1f-fabae854c4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:47.147814 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-c1f93868-6af7-4c30-8a1f-fabae854c4a3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:47.148195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 522/1017] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:31:46 2022] POST /identity/v3/endpoints => generated 334 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:31:47.149076 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c1f93868-6af7-4c30-8a1f-fabae854c4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:47.253843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 496/1018] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:47.557501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc348455-de63-4eab-8e2f-e53a429f8a9f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:47.661869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:47.706607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc348455-de63-4eab-8e2f-e53a429f8a9f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:47.749175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2fcc2-f8f6-4005-9479-eb8e1c398b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:47.750126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2fcc2-f8f6-4005-9479-eb8e1c398b05 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:47.750643 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2fcc2-f8f6-4005-9479-eb8e1c398b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:47.751096 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2fcc2-f8f6-4005-9479-eb8e1c398b05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:47.752516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2fcc2-f8f6-4005-9479-eb8e1c398b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:47.754161 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2fcc2-f8f6-4005-9479-eb8e1c398b05 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:47.756043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 523/1019] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:47.766704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f93868-6af7-4c30-8a1f-fabae854c4a3 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:47.811196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:47.923668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f93868-6af7-4c30-8a1f-fabae854c4a3 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:47.966097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b8126-a015-47dc-8c9b-ca6fb777851a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:47.966731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b8126-a015-47dc-8c9b-ca6fb777851a None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:47.967010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b8126-a015-47dc-8c9b-ca6fb777851a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:47.967297 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b8126-a015-47dc-8c9b-ca6fb777851a None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:47.967942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b8126-a015-47dc-8c9b-ca6fb777851a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:47.969376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b8126-a015-47dc-8c9b-ca6fb777851a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:48.100308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 497/1020] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:31:47 2022] POST /identity/v3/projects => generated 311 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:31:49.908399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6e5ee-4541-488c-a8f1-6bf9cf4bde80 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:49.909280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6e5ee-4541-488c-a8f1-6bf9cf4bde80 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:49.909280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6e5ee-4541-488c-a8f1-6bf9cf4bde80 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:49.910313 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 498/1021] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:49.910861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87139b23-ee54-478b-a9a9-bd493c33fee5 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:49.911394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87139b23-ee54-478b-a9a9-bd493c33fee5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:49.911980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87139b23-ee54-478b-a9a9-bd493c33fee5 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:49.914843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 524/1022] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:49.916074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506d62e-7d71-4baa-a892-1b3aeea3c04d None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:49.916408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506d62e-7d71-4baa-a892-1b3aeea3c04d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:49.916623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506d62e-7d71-4baa-a892-1b3aeea3c04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:49.924357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550446d0-840e-431e-8996-12ed221fb6b7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:49.925040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550446d0-840e-431e-8996-12ed221fb6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:49.925556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550446d0-840e-431e-8996-12ed221fb6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:49.937927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-c506d62e-7d71-4baa-a892-1b3aeea3c04d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:49.939213 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c506d62e-7d71-4baa-a892-1b3aeea3c04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:49.961316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-550446d0-840e-431e-8996-12ed221fb6b7 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:49.963463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-550446d0-840e-431e-8996-12ed221fb6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:50.033559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 499/1023] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:50.075534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 525/1024] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:50.239428 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352892-8417-46ca-88bc-5bd09fafe99d None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:50.239667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352892-8417-46ca-88bc-5bd09fafe99d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:50.240007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352892-8417-46ca-88bc-5bd09fafe99d None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:50.241516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 500/1025] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:50.247792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75718864-6fb5-4616-8ee9-d26282757230 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:50.247792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75718864-6fb5-4616-8ee9-d26282757230 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:50.247792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75718864-6fb5-4616-8ee9-d26282757230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:50.270880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-75718864-6fb5-4616-8ee9-d26282757230 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:50.272152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-75718864-6fb5-4616-8ee9-d26282757230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:50.309100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad352892-8417-46ca-88bc-5bd09fafe99d None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:50.338066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:50.354514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:50.367162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 526/1026] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:50.368601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75718864-6fb5-4616-8ee9-d26282757230 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:50.460903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad352892-8417-46ca-88bc-5bd09fafe99d None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:50.501228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf73ec5-ad1a-49dd-bde2-68cf1ec03617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:50.501758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf73ec5-ad1a-49dd-bde2-68cf1ec03617 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:50.502035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf73ec5-ad1a-49dd-bde2-68cf1ec03617 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:50.502337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf73ec5-ad1a-49dd-bde2-68cf1ec03617 None admin] PATH_INFO: `/v3/services/image` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:50.503026 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf73ec5-ad1a-49dd-bde2-68cf1ec03617 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:50.504529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf73ec5-ad1a-49dd-bde2-68cf1ec03617 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:50.513125 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75718864-6fb5-4616-8ee9-d26282757230 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-7bf73ec5-ad1a-49dd-bde2-68cf1ec03617 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:50.514410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:50.515925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:50.517627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 501/1027] 10.222.0.58 () {56 vars in 1105 bytes} [Mon Aug 29 23:31:50 2022] GET /identity/v3/services/image => generated 86 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:50.521883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf73ec5-ad1a-49dd-bde2-68cf1ec03617 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:50.566505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c58a4-b9e4-4cde-9161-d82cecc9fd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:50.567145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c58a4-b9e4-4cde-9161-d82cecc9fd61 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:50.567421 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c58a4-b9e4-4cde-9161-d82cecc9fd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:50.567685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c58a4-b9e4-4cde-9161-d82cecc9fd61 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:50.581845 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c58a4-b9e4-4cde-9161-d82cecc9fd61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1a3c58a4-b9e4-4cde-9161-d82cecc9fd61 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:50.586776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:50.587786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:50.588653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 527/1028] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Aug 29 23:31:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:50.596088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c58a4-b9e4-4cde-9161-d82cecc9fd61 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:50.645804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:50.670799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf73ec5-ad1a-49dd-bde2-68cf1ec03617 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:50.674178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:50.710882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2779b9-3d14-4255-a104-c073fd107fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:50.711492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2779b9-3d14-4255-a104-c073fd107fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:50.711769 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2779b9-3d14-4255-a104-c073fd107fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:50.712038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2779b9-3d14-4255-a104-c073fd107fc6 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:50.712668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2779b9-3d14-4255-a104-c073fd107fc6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:50.712974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2779b9-3d14-4255-a104-c073fd107fc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:50.714573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2779b9-3d14-4255-a104-c073fd107fc6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:50.722249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 502/1029] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:50.722249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2779b9-3d14-4255-a104-c073fd107fc6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:50.727706 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c58a4-b9e4-4cde-9161-d82cecc9fd61 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:50.768560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab5c3a-d8d2-42a4-a098-fd7d42f9f40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:50.769084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab5c3a-d8d2-42a4-a098-fd7d42f9f40d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:50.769408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab5c3a-d8d2-42a4-a098-fd7d42f9f40d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:50.769683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab5c3a-d8d2-42a4-a098-fd7d42f9f40d None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:50.770350 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab5c3a-d8d2-42a4-a098-fd7d42f9f40d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:50.770684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab5c3a-d8d2-42a4-a098-fd7d42f9f40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:50.772084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab5c3a-d8d2-42a4-a098-fd7d42f9f40d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:50.783245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 528/1030] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Aug 29 23:31:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:50.784005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab5c3a-d8d2-42a4-a098-fd7d42f9f40d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:50.847029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2779b9-3d14-4255-a104-c073fd107fc6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:50.885510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338055a3-fb17-47dc-ba83-6c50973b7e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:50.886771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338055a3-fb17-47dc-ba83-6c50973b7e56 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:50.887230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338055a3-fb17-47dc-ba83-6c50973b7e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:50.887705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338055a3-fb17-47dc-ba83-6c50973b7e56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:50.889152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338055a3-fb17-47dc-ba83-6c50973b7e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:50.890618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338055a3-fb17-47dc-ba83-6c50973b7e56 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:50.892507 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 503/1031] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:50.893300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338055a3-fb17-47dc-ba83-6c50973b7e56 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:50.913819 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:50.915817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab5c3a-d8d2-42a4-a098-fd7d42f9f40d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:50.935014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:50.954181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed710e4-58b4-4448-8161-832606e4a09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:50.954819 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed710e4-58b4-4448-8161-832606e4a09c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:50.955085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed710e4-58b4-4448-8161-832606e4a09c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:50.955355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed710e4-58b4-4448-8161-832606e4a09c None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:50.955989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed710e4-58b4-4448-8161-832606e4a09c None admin] RBAC: Adding query filter params (type=image) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:50.956284 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed710e4-58b4-4448-8161-832606e4a09c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:50.957661 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed710e4-58b4-4448-8161-832606e4a09c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:50.963083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 529/1032] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:50.963805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed710e4-58b4-4448-8161-832606e4a09c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:51.011463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338055a3-fb17-47dc-ba83-6c50973b7e56 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:51.053236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfd71b-b9a4-40d2-9977-c386ada25dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:51.053806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfd71b-b9a4-40d2-9977-c386ada25dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:51.054076 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfd71b-b9a4-40d2-9977-c386ada25dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:51.054351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfd71b-b9a4-40d2-9977-c386ada25dc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:51.066274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfd71b-b9a4-40d2-9977-c386ada25dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:51.067768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfd71b-b9a4-40d2-9977-c386ada25dc6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-47cfd71b-b9a4-40d2-9977-c386ada25dc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:51.084425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:51.085499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:51.086446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:51.086446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:51.086446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:51.086446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 504/1033] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:51.086697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfd71b-b9a4-40d2-9977-c386ada25dc6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:51.094400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed710e4-58b4-4448-8161-832606e4a09c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:51.132142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a8a1e-9ae3-4d81-83ee-d0413af39a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:51.132747 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a8a1e-9ae3-4d81-83ee-d0413af39a81 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:51.133028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a8a1e-9ae3-4d81-83ee-d0413af39a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:51.133291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a8a1e-9ae3-4d81-83ee-d0413af39a81 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:51.133940 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a8a1e-9ae3-4d81-83ee-d0413af39a81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:51.135431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a8a1e-9ae3-4d81-83ee-d0413af39a81 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:51.164472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:51.204539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfd71b-b9a4-40d2-9977-c386ada25dc6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:51.239054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bb1bf-2c5f-40e5-8454-e07deb5792de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:51.239680 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bb1bf-2c5f-40e5-8454-e07deb5792de None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:51.244644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bb1bf-2c5f-40e5-8454-e07deb5792de None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:51.244644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bb1bf-2c5f-40e5-8454-e07deb5792de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:51.244644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bb1bf-2c5f-40e5-8454-e07deb5792de None admin] RBAC: Adding query filter params (interface=public, service_id=07f309e060c84069a8d69e959e723c3f, region_id=RegionOne) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:51.244644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bb1bf-2c5f-40e5-8454-e07deb5792de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:51.244644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bb1bf-2c5f-40e5-8454-e07deb5792de None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:51.248481 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 505/1034] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Aug 29 23:31:51 2022] GET /identity/v3/endpoints?service_id=07f309e060c84069a8d69e959e723c3f&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:51.249387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bb1bf-2c5f-40e5-8454-e07deb5792de None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:51.271082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 530/1035] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:31:50 2022] POST /identity/v3/users => generated 285 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:31:51.272107 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a8a1e-9ae3-4d81-83ee-d0413af39a81 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:51.329479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:51.426265 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bb1bf-2c5f-40e5-8454-e07deb5792de None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:51.430205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a8a1e-9ae3-4d81-83ee-d0413af39a81 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:51.471401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:51.480903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3c17b-3ac4-4074-9201-10de9e7a14fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:51.481133 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95baa9-8e95-42d0-ac15-b147a0cad83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:51.481544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95baa9-8e95-42d0-ac15-b147a0cad83e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:51.481714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c17b-3ac4-4074-9201-10de9e7a14fd None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:51.481910 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95baa9-8e95-42d0-ac15-b147a0cad83e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:51.482089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c17b-3ac4-4074-9201-10de9e7a14fd None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:51.482190 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95baa9-8e95-42d0-ac15-b147a0cad83e None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:51.482534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c17b-3ac4-4074-9201-10de9e7a14fd None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:51.483219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95baa9-8e95-42d0-ac15-b147a0cad83e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:51.483636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c17b-3ac4-4074-9201-10de9e7a14fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:51.484078 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c17b-3ac4-4074-9201-10de9e7a14fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:51.484305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95baa9-8e95-42d0-ac15-b147a0cad83e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:51.492330 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c17b-3ac4-4074-9201-10de9e7a14fd None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:51.492471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 531/1036] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Aug 29 23:31:51 2022] GET /identity/v3/services => generated 1957 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:31:51.500506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 506/1037] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:51.507428 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95baa9-8e95-42d0-ac15-b147a0cad83e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:51.654406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:51.659465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95baa9-8e95-42d0-ac15-b147a0cad83e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:51.710117 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1509a-9f6f-4cd8-96e4-1770a50835b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:51.711128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1509a-9f6f-4cd8-96e4-1770a50835b2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:51.712071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1509a-9f6f-4cd8-96e4-1770a50835b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:51.712956 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1509a-9f6f-4cd8-96e4-1770a50835b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:51.725814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1509a-9f6f-4cd8-96e4-1770a50835b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:51.727413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1509a-9f6f-4cd8-96e4-1770a50835b2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-54a1509a-9f6f-4cd8-96e4-1770a50835b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:51.741392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:51.742631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:51.743731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:51.743731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:51.743731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:51.743731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:51.743731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:51.743731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:51.743731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:51.743731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:51.743731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:31:51.743731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:51.743731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 532/1038] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Aug 29 23:31:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:51.750730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3c17b-3ac4-4074-9201-10de9e7a14fd None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:51.859595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3c17b-3ac4-4074-9201-10de9e7a14fd None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:51.906479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0810dbf-b7ef-45db-bb04-7b0c2c7a4b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:51.907051 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0810dbf-b7ef-45db-bb04-7b0c2c7a4b42 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:51.907338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0810dbf-b7ef-45db-bb04-7b0c2c7a4b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:51.907682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0810dbf-b7ef-45db-bb04-7b0c2c7a4b42 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:51.909175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0810dbf-b7ef-45db-bb04-7b0c2c7a4b42 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:51.909497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0810dbf-b7ef-45db-bb04-7b0c2c7a4b42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:51.911840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0810dbf-b7ef-45db-bb04-7b0c2c7a4b42 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:51.937027 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 507/1039] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Aug 29 23:31:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:51.943046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1509a-9f6f-4cd8-96e4-1770a50835b2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:52.069901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1509a-9f6f-4cd8-96e4-1770a50835b2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:52.110998 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0464d77-9d4a-45f9-a367-a6c90db23df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:52.111615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0464d77-9d4a-45f9-a367-a6c90db23df1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:52.111890 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0464d77-9d4a-45f9-a367-a6c90db23df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:52.112158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0464d77-9d4a-45f9-a367-a6c90db23df1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:52.144384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:52.197945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0464d77-9d4a-45f9-a367-a6c90db23df1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:52.200858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0464d77-9d4a-45f9-a367-a6c90db23df1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:52.351653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 533/1040] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:31:51 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 23:31:52.354370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0810dbf-b7ef-45db-bb04-7b0c2c7a4b42 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:52.411811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:52.475163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0810dbf-b7ef-45db-bb04-7b0c2c7a4b42 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:52.513484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375be1bb-d487-4dc0-af6e-93ff49829c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:52.514480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375be1bb-d487-4dc0-af6e-93ff49829c74 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:52.514761 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375be1bb-d487-4dc0-af6e-93ff49829c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:52.515096 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375be1bb-d487-4dc0-af6e-93ff49829c74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:52.526280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375be1bb-d487-4dc0-af6e-93ff49829c74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:52.528350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375be1bb-d487-4dc0-af6e-93ff49829c74 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-375be1bb-d487-4dc0-af6e-93ff49829c74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:52.548101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:52.549159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:52.550740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:52.550740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:52.550740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:52.550740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 508/1041] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:52.555216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0464d77-9d4a-45f9-a367-a6c90db23df1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:52.628294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:52.688924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0464d77-9d4a-45f9-a367-a6c90db23df1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:52.742930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1ea1e-90db-4408-b20e-546514270832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:52.743757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1ea1e-90db-4408-b20e-546514270832 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:52.744176 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1ea1e-90db-4408-b20e-546514270832 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:52.744583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1ea1e-90db-4408-b20e-546514270832 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:52.746026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1ea1e-90db-4408-b20e-546514270832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:52.746700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1ea1e-90db-4408-b20e-546514270832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:52.748232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1ea1e-90db-4408-b20e-546514270832 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:52.761714 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 534/1042] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:52.768556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375be1bb-d487-4dc0-af6e-93ff49829c74 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:52.890175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:52.893977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375be1bb-d487-4dc0-af6e-93ff49829c74 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:52.943295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460160e-644e-4ce6-a513-73abe72ed4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:52.944270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460160e-644e-4ce6-a513-73abe72ed4df None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:52.944751 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460160e-644e-4ce6-a513-73abe72ed4df None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:52.945300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460160e-644e-4ce6-a513-73abe72ed4df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:52.959157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460160e-644e-4ce6-a513-73abe72ed4df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:52.961749 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460160e-644e-4ce6-a513-73abe72ed4df None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-9460160e-644e-4ce6-a513-73abe72ed4df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:52.984508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:52.985793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:52.993811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:52.993811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:52.993811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:52.993811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:52.993811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:52.993811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:52.993811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:52.993811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:52.993811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 509/1043] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:52.994340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1ea1e-90db-4408-b20e-546514270832 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:53.155198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1ea1e-90db-4408-b20e-546514270832 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:53.179941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:53.196227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7af74-f0ce-4789-88df-dc6c48186abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:53.196930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7af74-f0ce-4789-88df-dc6c48186abc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:53.197268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7af74-f0ce-4789-88df-dc6c48186abc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:53.197596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7af74-f0ce-4789-88df-dc6c48186abc None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:53.199020 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7af74-f0ce-4789-88df-dc6c48186abc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:53.199438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7af74-f0ce-4789-88df-dc6c48186abc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:53.200777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7af74-f0ce-4789-88df-dc6c48186abc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:53.222464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 535/1044] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:53.225836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460160e-644e-4ce6-a513-73abe72ed4df None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:53.364360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460160e-644e-4ce6-a513-73abe72ed4df None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:53.428265 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34d7ee-9df0-487f-aedd-6173ee3b4f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:53.429319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34d7ee-9df0-487f-aedd-6173ee3b4f24 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:53.429808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34d7ee-9df0-487f-aedd-6173ee3b4f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:53.430261 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34d7ee-9df0-487f-aedd-6173ee3b4f24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:53.431641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34d7ee-9df0-487f-aedd-6173ee3b4f24 None admin] RBAC: Adding query filter params (role.id=c6a4683645ce4b68a14f2edeee414d92, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=e4a8261d92064b838fc6e8f415879566) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:53.432192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34d7ee-9df0-487f-aedd-6173ee3b4f24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:53.435653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34d7ee-9df0-487f-aedd-6173ee3b4f24 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:53.445624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f363b76-fd22-4c09-bf09-89469c74c3b5 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:53.445923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f363b76-fd22-4c09-bf09-89469c74c3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:53.446280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f363b76-fd22-4c09-bf09-89469c74c3b5 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:53.446536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 510/1045] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:31:53 2022] GET /identity/v3/role_assignments?user.id=e4a8261d92064b838fc6e8f415879566&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=c6a4683645ce4b68a14f2edeee414d92 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:53.447672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 536/1046] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:31:53.454789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff112da-6527-4750-b2d6-b0aa26b1b62a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:53.455262 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff112da-6527-4750-b2d6-b0aa26b1b62a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:53.455698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff112da-6527-4750-b2d6-b0aa26b1b62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:53.485142 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-0ff112da-6527-4750-b2d6-b0aa26b1b62a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:53.486377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-0ff112da-6527-4750-b2d6-b0aa26b1b62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:53.507383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:53.513433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f359e-fbfe-45a3-b484-7d7521175e68 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:53.514031 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f359e-fbfe-45a3-b484-7d7521175e68 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:53.514255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f359e-fbfe-45a3-b484-7d7521175e68 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:53.516019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 537/1047] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:53.521470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4623f-f4c8-4330-b772-52a13591a8a3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:53.521757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4623f-f4c8-4330-b772-52a13591a8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:53.522028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4623f-f4c8-4330-b772-52a13591a8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:53.542482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5bc4623f-f4c8-4330-b772-52a13591a8a3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:53.543749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5bc4623f-f4c8-4330-b772-52a13591a8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:53.567476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 511/1048] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:53.627819 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 538/1049] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:31:53.841613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff112da-6527-4750-b2d6-b0aa26b1b62a None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:53.903561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4623f-f4c8-4330-b772-52a13591a8a3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:53.965442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff112da-6527-4750-b2d6-b0aa26b1b62a None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:54.024434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6839e99-d7db-4676-b33a-13f8c928dc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:54.025746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6839e99-d7db-4676-b33a-13f8c928dc58 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:54.026287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6839e99-d7db-4676-b33a-13f8c928dc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:54.026775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6839e99-d7db-4676-b33a-13f8c928dc58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:54.048487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6839e99-d7db-4676-b33a-13f8c928dc58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:54.058682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4623f-f4c8-4330-b772-52a13591a8a3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-f6839e99-d7db-4676-b33a-13f8c928dc58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:54.059515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:54.060460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:31:54.061316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:54.062556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 512/1050] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:31:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:54.071057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6839e99-d7db-4676-b33a-13f8c928dc58 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:54.079561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:54.097682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:54.116533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fa367-c338-4977-9413-b9193dfb128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:54.117527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fa367-c338-4977-9413-b9193dfb128c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:54.117988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fa367-c338-4977-9413-b9193dfb128c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:54.118457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fa367-c338-4977-9413-b9193dfb128c None admin] PATH_INFO: `/v3/services/image` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:54.119351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fa367-c338-4977-9413-b9193dfb128c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:54.123381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fa367-c338-4977-9413-b9193dfb128c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c19fa367-c338-4977-9413-b9193dfb128c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:54.138292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:54.139718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:54.140772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 539/1051] 10.222.0.58 () {56 vars in 1105 bytes} [Mon Aug 29 23:31:53 2022] GET /identity/v3/services/image => generated 86 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:54.148774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fa367-c338-4977-9413-b9193dfb128c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:54.240289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6839e99-d7db-4676-b33a-13f8c928dc58 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:54.294725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6729bbc-a7d2-4bc1-af48-225c231bcf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:54.295712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6729bbc-a7d2-4bc1-af48-225c231bcf76 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:54.296125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6729bbc-a7d2-4bc1-af48-225c231bcf76 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:54.296530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6729bbc-a7d2-4bc1-af48-225c231bcf76 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:54.297558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6729bbc-a7d2-4bc1-af48-225c231bcf76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:54.298025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6729bbc-a7d2-4bc1-af48-225c231bcf76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:54.300033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6729bbc-a7d2-4bc1-af48-225c231bcf76 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:54.309416 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fa367-c338-4977-9413-b9193dfb128c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:54.316804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 513/1052] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:54.318829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6729bbc-a7d2-4bc1-af48-225c231bcf76 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:54.353969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bf56f-bbf2-458d-8a72-275cf4a58e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:54.355623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bf56f-bbf2-458d-8a72-275cf4a58e72 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:54.355898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bf56f-bbf2-458d-8a72-275cf4a58e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:54.356182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bf56f-bbf2-458d-8a72-275cf4a58e72 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:54.356813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bf56f-bbf2-458d-8a72-275cf4a58e72 None admin] RBAC: Adding query filter params (name=image) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:54.357113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bf56f-bbf2-458d-8a72-275cf4a58e72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:54.358560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bf56f-bbf2-458d-8a72-275cf4a58e72 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:54.369710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 540/1053] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:54.371157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bf56f-bbf2-458d-8a72-275cf4a58e72 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:54.434668 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:54.447218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:54.478911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6729bbc-a7d2-4bc1-af48-225c231bcf76 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:54.518672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bf56f-bbf2-458d-8a72-275cf4a58e72 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:54.520504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb48482-f6e8-4888-aece-a19fbfad3ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:54.521149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48482-f6e8-4888-aece-a19fbfad3ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:54.521427 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48482-f6e8-4888-aece-a19fbfad3ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:54.521767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48482-f6e8-4888-aece-a19fbfad3ab8 None admin] PATH_INFO: `/v3/users/07fdcf436b6045379290daf80d0c3199` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:54.561541 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb48482-f6e8-4888-aece-a19fbfad3ab8 None admin] RBAC: Authorizing `identity:get_user(user_id=07fdcf436b6045379290daf80d0c3199)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:54.563632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb48482-f6e8-4888-aece-a19fbfad3ab8 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:54.565850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 514/1054] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:31:54 2022] GET /identity/v3/users/07fdcf436b6045379290daf80d0c3199 => generated 285 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:54.573199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb48482-f6e8-4888-aece-a19fbfad3ab8 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:54.588971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61161f-b6a6-4285-8652-5845fe92c0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:54.590220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61161f-b6a6-4285-8652-5845fe92c0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:54.591231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61161f-b6a6-4285-8652-5845fe92c0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:54.592022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61161f-b6a6-4285-8652-5845fe92c0f6 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:54.592922 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61161f-b6a6-4285-8652-5845fe92c0f6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:54.593639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61161f-b6a6-4285-8652-5845fe92c0f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:54.595664 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61161f-b6a6-4285-8652-5845fe92c0f6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:54.605028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 541/1055] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:54.611514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61161f-b6a6-4285-8652-5845fe92c0f6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:54.694197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb48482-f6e8-4888-aece-a19fbfad3ab8 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:54.747583 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749eaad-4e85-4b62-b83c-6e0ce1f9df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:54.748483 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749eaad-4e85-4b62-b83c-6e0ce1f9df60 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:54.748938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749eaad-4e85-4b62-b83c-6e0ce1f9df60 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:54.749702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749eaad-4e85-4b62-b83c-6e0ce1f9df60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:54.753761 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:54.764857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61161f-b6a6-4285-8652-5845fe92c0f6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:54.767510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:54.817061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a5925-94a6-4108-8ae2-6ddd21fa75b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:54.818319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a5925-94a6-4108-8ae2-6ddd21fa75b6 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:54.818998 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a5925-94a6-4108-8ae2-6ddd21fa75b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:54.819587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a5925-94a6-4108-8ae2-6ddd21fa75b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:54.820731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a5925-94a6-4108-8ae2-6ddd21fa75b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:54.824130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a5925-94a6-4108-8ae2-6ddd21fa75b6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:54.855932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749eaad-4e85-4b62-b83c-6e0ce1f9df60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:54.857403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749eaad-4e85-4b62-b83c-6e0ce1f9df60 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:54.905366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 542/1056] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:31:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:31:54.994121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 515/1057] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:31:54 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 23:31:54.996537 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a5925-94a6-4108-8ae2-6ddd21fa75b6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:55.185766 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a5925-94a6-4108-8ae2-6ddd21fa75b6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:55.225223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db322331-9b45-47fd-b2e1-947edf1bcba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:55.225855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db322331-9b45-47fd-b2e1-947edf1bcba8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:55.226133 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db322331-9b45-47fd-b2e1-947edf1bcba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:55.226624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db322331-9b45-47fd-b2e1-947edf1bcba8 None admin] PATH_INFO: `/v3/projects/3ca9a513f6144fe6aa717b06782d83f2` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:55.227860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db322331-9b45-47fd-b2e1-947edf1bcba8 None admin] RBAC: Authorizing `identity:get_project(project_id=3ca9a513f6144fe6aa717b06782d83f2)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:55.229386 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db322331-9b45-47fd-b2e1-947edf1bcba8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:55.231536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 543/1058] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:31:54 2022] GET /identity/v3/projects/3ca9a513f6144fe6aa717b06782d83f2 => generated 311 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:55.238171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749eaad-4e85-4b62-b83c-6e0ce1f9df60 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:55.280214 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:55.368812 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749eaad-4e85-4b62-b83c-6e0ce1f9df60 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:55.408530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500de50c-917d-41c3-a4cd-46d7337e909c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:55.409143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500de50c-917d-41c3-a4cd-46d7337e909c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:55.409429 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500de50c-917d-41c3-a4cd-46d7337e909c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:55.409685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500de50c-917d-41c3-a4cd-46d7337e909c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:55.410409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500de50c-917d-41c3-a4cd-46d7337e909c None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=3ca9a513f6144fe6aa717b06782d83f2, user.id=07fdcf436b6045379290daf80d0c3199) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:55.410737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500de50c-917d-41c3-a4cd-46d7337e909c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:55.412198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500de50c-917d-41c3-a4cd-46d7337e909c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:55.426487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 516/1059] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:31:55 2022] GET /identity/v3/role_assignments?user.id=07fdcf436b6045379290daf80d0c3199&scope.project.id=3ca9a513f6144fe6aa717b06782d83f2&role.id=d86dba8efe20423abdde06031690beb8 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:55.503251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db393b3-c23c-47b1-a8c3-43dfdfce8642 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:55.503451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db393b3-c23c-47b1-a8c3-43dfdfce8642 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:55.503822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db393b3-c23c-47b1-a8c3-43dfdfce8642 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:55.505304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 544/1060] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:31:55.511421 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28144ae-a1cc-4548-91f8-5b90a722513e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:55.511718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28144ae-a1cc-4548-91f8-5b90a722513e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:55.511981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28144ae-a1cc-4548-91f8-5b90a722513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:55.536217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d28144ae-a1cc-4548-91f8-5b90a722513e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:55.537753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d28144ae-a1cc-4548-91f8-5b90a722513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:55.616044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 517/1061] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:55.889486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db393b3-c23c-47b1-a8c3-43dfdfce8642 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:55.935523 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:56.087177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db393b3-c23c-47b1-a8c3-43dfdfce8642 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:56.155814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342b6c1-7bda-4235-8ee5-97e1bfdd9862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:56.156758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b6c1-7bda-4235-8ee5-97e1bfdd9862 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:56.157238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b6c1-7bda-4235-8ee5-97e1bfdd9862 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:56.157719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b6c1-7bda-4235-8ee5-97e1bfdd9862 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:56.178348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342b6c1-7bda-4235-8ee5-97e1bfdd9862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d342b6c1-7bda-4235-8ee5-97e1bfdd9862 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:56.188880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:56.191533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:56.195720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 545/1062] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Aug 29 23:31:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:56.201897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28144ae-a1cc-4548-91f8-5b90a722513e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:56.251628 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:56.373869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28144ae-a1cc-4548-91f8-5b90a722513e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:56.438047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6efe96-5005-41d3-b5a2-3e58c1f8c686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:56.438995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6efe96-5005-41d3-b5a2-3e58c1f8c686 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:56.439440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6efe96-5005-41d3-b5a2-3e58c1f8c686 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:56.439908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6efe96-5005-41d3-b5a2-3e58c1f8c686 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:56.441066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6efe96-5005-41d3-b5a2-3e58c1f8c686 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:56.441596 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6efe96-5005-41d3-b5a2-3e58c1f8c686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:56.444162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6efe96-5005-41d3-b5a2-3e58c1f8c686 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:56.464606 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 518/1063] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Aug 29 23:31:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:56.466746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342b6c1-7bda-4235-8ee5-97e1bfdd9862 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:56.615849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:56.652847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342b6c1-7bda-4235-8ee5-97e1bfdd9862 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:56.717958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d09c8-e484-4776-b0f1-bc77020db778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:56.718986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d09c8-e484-4776-b0f1-bc77020db778 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:56.719468 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d09c8-e484-4776-b0f1-bc77020db778 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:56.720021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d09c8-e484-4776-b0f1-bc77020db778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:56.739820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d09c8-e484-4776-b0f1-bc77020db778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:56.742470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d09c8-e484-4776-b0f1-bc77020db778 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-255d09c8-e484-4776-b0f1-bc77020db778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:56.770738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:56.771817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:56.777767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:56.777767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:56.777767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:56.777767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 546/1064] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:56.780438 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6efe96-5005-41d3-b5a2-3e58c1f8c686 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:56.942549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:56.954280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6efe96-5005-41d3-b5a2-3e58c1f8c686 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:56.994320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f54607-d62b-4cd8-b413-fb4588cf1603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:56.995439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f54607-d62b-4cd8-b413-fb4588cf1603 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:56.995710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f54607-d62b-4cd8-b413-fb4588cf1603 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:56.995985 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f54607-d62b-4cd8-b413-fb4588cf1603 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:56.996641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f54607-d62b-4cd8-b413-fb4588cf1603 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:56.996941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f54607-d62b-4cd8-b413-fb4588cf1603 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:56.998323 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f54607-d62b-4cd8-b413-fb4588cf1603 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:57.015754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 519/1065] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:57.021432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d09c8-e484-4776-b0f1-bc77020db778 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:57.145513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d09c8-e484-4776-b0f1-bc77020db778 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:57.183224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5066a3-1cf3-4617-ab32-48d316099d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:57.183858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5066a3-1cf3-4617-ab32-48d316099d34 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:57.184137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5066a3-1cf3-4617-ab32-48d316099d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:57.184405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5066a3-1cf3-4617-ab32-48d316099d34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:57.186598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:57.194035 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5066a3-1cf3-4617-ab32-48d316099d34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:57.195559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5066a3-1cf3-4617-ab32-48d316099d34 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ba5066a3-1cf3-4617-ab32-48d316099d34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:31:57.208424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:57.210244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:57.212780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:57.212780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:57.212780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:31:57.212780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:31:57.212780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:31:57.212780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:31:57.212780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:31:57.212780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:31:57.212780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:31:57.212780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:57.215183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 547/1066] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Aug 29 23:31:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:57.217319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f54607-d62b-4cd8-b413-fb4588cf1603 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:57.337922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f54607-d62b-4cd8-b413-fb4588cf1603 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:57.379709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7c34e-84e1-4417-a9bf-0f04d42e1d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:57.380335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c34e-84e1-4417-a9bf-0f04d42e1d43 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:57.380615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c34e-84e1-4417-a9bf-0f04d42e1d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:57.380888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c34e-84e1-4417-a9bf-0f04d42e1d43 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:57.381638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c34e-84e1-4417-a9bf-0f04d42e1d43 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:57.381949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c34e-84e1-4417-a9bf-0f04d42e1d43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:57.383448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c34e-84e1-4417-a9bf-0f04d42e1d43 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:57.404550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 520/1067] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Aug 29 23:31:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:57.410501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5066a3-1cf3-4617-ab32-48d316099d34 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:57.531320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5066a3-1cf3-4617-ab32-48d316099d34 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:57.573300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd936f-3eac-4d6b-8fe4-df79d6151e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:57.574006 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd936f-3eac-4d6b-8fe4-df79d6151e84 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:57.574292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd936f-3eac-4d6b-8fe4-df79d6151e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:57.574608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd936f-3eac-4d6b-8fe4-df79d6151e84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:57.588550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd936f-3eac-4d6b-8fe4-df79d6151e84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:57.589943 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd936f-3eac-4d6b-8fe4-df79d6151e84 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ecbd936f-3eac-4d6b-8fe4-df79d6151e84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:57.606901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:57.608662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:57.611143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:57.611143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:57.611143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:31:57.612016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 548/1068] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:31:57.614726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7c34e-84e1-4417-a9bf-0f04d42e1d43 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:57.628778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:57.687425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccf4a0-ee95-46b8-ada6-b4584a8ebce1 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:57.687928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccf4a0-ee95-46b8-ada6-b4584a8ebce1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:57.688411 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccf4a0-ee95-46b8-ada6-b4584a8ebce1 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:57.694770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 549/1069] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:57.697128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3727e40-b48a-4e24-bd23-e819433e2e93 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:57.697649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3727e40-b48a-4e24-bd23-e819433e2e93 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:57.698216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3727e40-b48a-4e24-bd23-e819433e2e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:57.720396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e3727e40-b48a-4e24-bd23-e819433e2e93 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:57.721626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e3727e40-b48a-4e24-bd23-e819433e2e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:57.754227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7c34e-84e1-4417-a9bf-0f04d42e1d43 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:57.790293 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:57.797507 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fed96-f4df-4c5c-b1c8-9d2b9f311b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:57.798135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fed96-f4df-4c5c-b1c8-9d2b9f311b9e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:57.798454 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fed96-f4df-4c5c-b1c8-9d2b9f311b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:57.798730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fed96-f4df-4c5c-b1c8-9d2b9f311b9e None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:57.799397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fed96-f4df-4c5c-b1c8-9d2b9f311b9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:57.802244 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fed96-f4df-4c5c-b1c8-9d2b9f311b9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:57.802244 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fed96-f4df-4c5c-b1c8-9d2b9f311b9e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:57.816858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 521/1070] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:57.823048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fed96-f4df-4c5c-b1c8-9d2b9f311b9e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:57.847028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 550/1071] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:57.947704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:58.010316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fed96-f4df-4c5c-b1c8-9d2b9f311b9e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:58.069047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a5b89-5e93-46bd-abcf-cf7190c6ab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:58.069999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a5b89-5e93-46bd-abcf-cf7190c6ab71 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:58.070438 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a5b89-5e93-46bd-abcf-cf7190c6ab71 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:58.070886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a5b89-5e93-46bd-abcf-cf7190c6ab71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:58.083528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a5b89-5e93-46bd-abcf-cf7190c6ab71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:58.085882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a5b89-5e93-46bd-abcf-cf7190c6ab71 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-fe9a5b89-5e93-46bd-abcf-cf7190c6ab71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:31:58.107352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:58.108389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:58.114415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:31:58.114415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:31:58.114415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:31:58.114415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:31:58.114415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:31:58.114415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:31:58.114415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:31:58.114415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:58.114415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 522/1072] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:31:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:58.117171 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3727e40-b48a-4e24-bd23-e819433e2e93 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:58.218612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a5b89-5e93-46bd-abcf-cf7190c6ab71 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:58.275322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3727e40-b48a-4e24-bd23-e819433e2e93 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:58.320831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570898d7-2fa3-42bf-8606-be9fafe3076e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:58.322067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570898d7-2fa3-42bf-8606-be9fafe3076e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:58.323651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570898d7-2fa3-42bf-8606-be9fafe3076e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:58.324657 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570898d7-2fa3-42bf-8606-be9fafe3076e None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:58.325914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570898d7-2fa3-42bf-8606-be9fafe3076e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:58.326470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570898d7-2fa3-42bf-8606-be9fafe3076e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:58.329128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570898d7-2fa3-42bf-8606-be9fafe3076e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:58.349518 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a5b89-5e93-46bd-abcf-cf7190c6ab71 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:58.352856 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 551/1073] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:31:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:58.355651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69650f-34ba-4a57-8aad-654854d2492b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:58.356157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69650f-34ba-4a57-8aad-654854d2492b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:58.356643 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69650f-34ba-4a57-8aad-654854d2492b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:58.364595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 552/1074] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:31:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:31:58.364595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69650f-34ba-4a57-8aad-654854d2492b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:58.382689 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:58.393226 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7c467-3f71-4994-afbd-d6f77d0880a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:58.393993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7c467-3f71-4994-afbd-d6f77d0880a6 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:58.394495 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7c467-3f71-4994-afbd-d6f77d0880a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:58.394954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7c467-3f71-4994-afbd-d6f77d0880a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:58.412543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:58.480164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a7c467-3f71-4994-afbd-d6f77d0880a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:58.482030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a7c467-3f71-4994-afbd-d6f77d0880a6 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:58.500000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69650f-34ba-4a57-8aad-654854d2492b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:58.546145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8dc237-c32b-4312-959c-347301e3988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:58.546835 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dc237-c32b-4312-959c-347301e3988a None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:58.547108 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dc237-c32b-4312-959c-347301e3988a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:58.547375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dc237-c32b-4312-959c-347301e3988a None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/e4a8261d92064b838fc6e8f415879566/roles/c6a4683645ce4b68a14f2edeee414d92` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:58.549730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8dc237-c32b-4312-959c-347301e3988a None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=e4a8261d92064b838fc6e8f415879566, role_id=c6a4683645ce4b68a14f2edeee414d92)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:58.551232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8dc237-c32b-4312-959c-347301e3988a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:58.624438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 553/1075] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:31:58 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/e4a8261d92064b838fc6e8f415879566/roles/c6a4683645ce4b68a14f2edeee414d92 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:31:58.625844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801531f-0082-4e2e-a79c-b1de411dd7e3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:58.630657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 523/1076] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:31:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 23:31:58.630794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801531f-0082-4e2e-a79c-b1de411dd7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:58.630794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801531f-0082-4e2e-a79c-b1de411dd7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:58.635093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7c467-3f71-4994-afbd-d6f77d0880a6 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:58.653519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9801531f-0082-4e2e-a79c-b1de411dd7e3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:31:58.655190 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9801531f-0082-4e2e-a79c-b1de411dd7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:31:58.672883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:58.758988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 554/1077] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:31:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:31:58.772934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7c467-3f71-4994-afbd-d6f77d0880a6 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:58.817522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d8bd2-e7cb-40e3-adc0-1e0a0c306823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:58.818174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d8bd2-e7cb-40e3-adc0-1e0a0c306823 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:58.818551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d8bd2-e7cb-40e3-adc0-1e0a0c306823 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:58.818827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d8bd2-e7cb-40e3-adc0-1e0a0c306823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:58.832587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d8bd2-e7cb-40e3-adc0-1e0a0c306823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:58.834156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d8bd2-e7cb-40e3-adc0-1e0a0c306823 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-678d8bd2-e7cb-40e3-adc0-1e0a0c306823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:31:58.852294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:58.862714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:31:58.863655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:31:58.863655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:31:58.863655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:58.863655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 524/1078] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:58.863909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801531f-0082-4e2e-a79c-b1de411dd7e3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:58.920934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:59.056998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d8bd2-e7cb-40e3-adc0-1e0a0c306823 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:59.063945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801531f-0082-4e2e-a79c-b1de411dd7e3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:59.129886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d1b29-d8d4-4372-829c-82083714dde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:59.130829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d1b29-d8d4-4372-829c-82083714dde9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:59.131258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d1b29-d8d4-4372-829c-82083714dde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:59.131661 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d1b29-d8d4-4372-829c-82083714dde9 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:59.132711 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d1b29-d8d4-4372-829c-82083714dde9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:59.133160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d1b29-d8d4-4372-829c-82083714dde9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:59.135679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d1b29-d8d4-4372-829c-82083714dde9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:59.162702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 555/1079] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:59.185787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:59.211263 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d8bd2-e7cb-40e3-adc0-1e0a0c306823 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:59.287698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca107-fc31-4603-a532-181e62e93d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:59.288813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca107-fc31-4603-a532-181e62e93d2f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:59.289342 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca107-fc31-4603-a532-181e62e93d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:59.289767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca107-fc31-4603-a532-181e62e93d2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:59.313942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aca107-fc31-4603-a532-181e62e93d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-c0aca107-fc31-4603-a532-181e62e93d2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:31:59.325638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:31:59.326738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:31:59.327614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:31:59.331998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 525/1080] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:31:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:31:59.334076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d1b29-d8d4-4372-829c-82083714dde9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:59.472093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:31:59.502621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d1b29-d8d4-4372-829c-82083714dde9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:59.571085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ece80f-a85d-477e-a8f6-f35fdb090750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:59.572086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece80f-a85d-477e-a8f6-f35fdb090750 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:59.572585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece80f-a85d-477e-a8f6-f35fdb090750 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:59.574581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece80f-a85d-477e-a8f6-f35fdb090750 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:59.575877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ece80f-a85d-477e-a8f6-f35fdb090750 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:31:59.576427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ece80f-a85d-477e-a8f6-f35fdb090750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:59.578989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ece80f-a85d-477e-a8f6-f35fdb090750 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:59.595833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 556/1081] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:31:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:59.602239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca107-fc31-4603-a532-181e62e93d2f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:59.713109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca107-fc31-4603-a532-181e62e93d2f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:59.750044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a71f6d-8c71-488b-a9fc-7859fabc1c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:59.750630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a71f6d-8c71-488b-a9fc-7859fabc1c3e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:59.750910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a71f6d-8c71-488b-a9fc-7859fabc1c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:59.751183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a71f6d-8c71-488b-a9fc-7859fabc1c3e None admin] PATH_INFO: `/v3/users/07fdcf436b6045379290daf80d0c3199` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:59.752322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a71f6d-8c71-488b-a9fc-7859fabc1c3e None admin] RBAC: Authorizing `identity:get_user(user_id=07fdcf436b6045379290daf80d0c3199)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:59.753761 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a71f6d-8c71-488b-a9fc-7859fabc1c3e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:59.755399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 526/1082] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:31:59 2022] GET /identity/v3/users/07fdcf436b6045379290daf80d0c3199 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:31:59.761379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ece80f-a85d-477e-a8f6-f35fdb090750 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:59.865354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ece80f-a85d-477e-a8f6-f35fdb090750 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:31:59.899833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85a0d4-94b0-46ba-8f1f-1425b4480af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:31:59.900350 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85a0d4-94b0-46ba-8f1f-1425b4480af0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:31:59.900617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85a0d4-94b0-46ba-8f1f-1425b4480af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:31:59.901023 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85a0d4-94b0-46ba-8f1f-1425b4480af0 None admin] PATH_INFO: `/v3/projects/3ca9a513f6144fe6aa717b06782d83f2` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:31:59.902088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85a0d4-94b0-46ba-8f1f-1425b4480af0 None admin] RBAC: Authorizing `identity:get_project(project_id=3ca9a513f6144fe6aa717b06782d83f2)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:31:59.903588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85a0d4-94b0-46ba-8f1f-1425b4480af0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:31:59.905225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 557/1083] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:31:59 2022] GET /identity/v3/projects/3ca9a513f6144fe6aa717b06782d83f2 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:31:59.911425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a71f6d-8c71-488b-a9fc-7859fabc1c3e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:31:59.923756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:00.020715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a71f6d-8c71-488b-a9fc-7859fabc1c3e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:00.057154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd987511-fd85-4908-b7c2-8d0e52fd4542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:00.057809 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd987511-fd85-4908-b7c2-8d0e52fd4542 None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:00.058083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd987511-fd85-4908-b7c2-8d0e52fd4542 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:00.058354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd987511-fd85-4908-b7c2-8d0e52fd4542 None admin] PATH_INFO: `/v3/projects/3ca9a513f6144fe6aa717b06782d83f2/users/07fdcf436b6045379290daf80d0c3199/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:00.060300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd987511-fd85-4908-b7c2-8d0e52fd4542 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ca9a513f6144fe6aa717b06782d83f2, user_id=07fdcf436b6045379290daf80d0c3199, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:00.061741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd987511-fd85-4908-b7c2-8d0e52fd4542 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:00.136345 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 527/1084] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:31:59 2022] PUT /identity/v3/projects/3ca9a513f6144fe6aa717b06782d83f2/users/07fdcf436b6045379290daf80d0c3199/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:32:01.519729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cb8fa-7d07-48a1-a7e7-754d606c153c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:01.520290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cb8fa-7d07-48a1-a7e7-754d606c153c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:01.520675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cb8fa-7d07-48a1-a7e7-754d606c153c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:01.523196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 558/1085] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:32:01.532543 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59517f92-9502-4edf-b34d-c2c8ac5b728f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:01.533084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59517f92-9502-4edf-b34d-c2c8ac5b728f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:01.533508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59517f92-9502-4edf-b34d-c2c8ac5b728f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:01.558704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-59517f92-9502-4edf-b34d-c2c8ac5b728f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:01.559920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-59517f92-9502-4edf-b34d-c2c8ac5b728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:01.631868 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:01.638983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 528/1086] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:32:01.911477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cb8fa-7d07-48a1-a7e7-754d606c153c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:01.933671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:02.024167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cb8fa-7d07-48a1-a7e7-754d606c153c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:02.059383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be129bb-f277-4ed1-8be2-7cab94208fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:02.060026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be129bb-f277-4ed1-8be2-7cab94208fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:02.060302 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be129bb-f277-4ed1-8be2-7cab94208fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:02.060570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be129bb-f277-4ed1-8be2-7cab94208fa3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:02.072013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be129bb-f277-4ed1-8be2-7cab94208fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-6be129bb-f277-4ed1-8be2-7cab94208fa3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:32:02.077658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:02.079082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 23:32:02.079942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:32:02.082595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 559/1087] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Aug 29 23:32:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:32:02.084943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59517f92-9502-4edf-b34d-c2c8ac5b728f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:02.094560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77306b8-e4c9-4088-b648-e775f5940b5d None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:02.094849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77306b8-e4c9-4088-b648-e775f5940b5d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:02.095076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77306b8-e4c9-4088-b648-e775f5940b5d None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:02.096419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 560/1088] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:32:02.101884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2998b93-9bbc-481a-ab28-ab16174668a3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:02.102157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2998b93-9bbc-481a-ab28-ab16174668a3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:02.102459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2998b93-9bbc-481a-ab28-ab16174668a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:02.120803 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d2998b93-9bbc-481a-ab28-ab16174668a3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:02.122003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d2998b93-9bbc-481a-ab28-ab16174668a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:02.198274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 561/1089] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:32:02.221162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59517f92-9502-4edf-b34d-c2c8ac5b728f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:02.258248 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b3130-47f4-44df-a0f3-02fe97efe6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:02.259006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b3130-47f4-44df-a0f3-02fe97efe6de None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:02.259283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b3130-47f4-44df-a0f3-02fe97efe6de None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:02.259553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b3130-47f4-44df-a0f3-02fe97efe6de None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:02.260768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b3130-47f4-44df-a0f3-02fe97efe6de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:02.261066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b3130-47f4-44df-a0f3-02fe97efe6de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:02.262661 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b3130-47f4-44df-a0f3-02fe97efe6de None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:02.271793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 529/1090] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Aug 29 23:32:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:02.278392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2998b93-9bbc-481a-ab28-ab16174668a3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:02.288069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:02.415158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2998b93-9bbc-481a-ab28-ab16174668a3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:02.452866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693414b2-428d-46b6-89b7-6ad22cdd5315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:02.453420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693414b2-428d-46b6-89b7-6ad22cdd5315 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:02.453698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693414b2-428d-46b6-89b7-6ad22cdd5315 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:02.453970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693414b2-428d-46b6-89b7-6ad22cdd5315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:02.465966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693414b2-428d-46b6-89b7-6ad22cdd5315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:02.467469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693414b2-428d-46b6-89b7-6ad22cdd5315 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:02.478913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b3130-47f4-44df-a0f3-02fe97efe6de None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-693414b2-428d-46b6-89b7-6ad22cdd5315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:32:02.483578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:02.484673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:32:02.485592 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:32:02.485592 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:32:02.485592 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:32:02.485592 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 562/1091] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:32:02.490895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693414b2-428d-46b6-89b7-6ad22cdd5315 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:02.548612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:02.576142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:02.594973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b3130-47f4-44df-a0f3-02fe97efe6de None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:02.629087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693414b2-428d-46b6-89b7-6ad22cdd5315 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:02.637318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94f742-e2a4-49c2-8015-06079037a57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:02.637954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94f742-e2a4-49c2-8015-06079037a57b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:02.638538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94f742-e2a4-49c2-8015-06079037a57b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:02.638799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94f742-e2a4-49c2-8015-06079037a57b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:02.649687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94f742-e2a4-49c2-8015-06079037a57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-7a94f742-e2a4-49c2-8015-06079037a57b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:32:02.655727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:02.656822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:32:02.657678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 530/1092] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:32:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:32:02.663116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94f742-e2a4-49c2-8015-06079037a57b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:02.668392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60409a9b-c603-40ac-afef-6da0d657df79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:02.669001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60409a9b-c603-40ac-afef-6da0d657df79 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:02.669274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60409a9b-c603-40ac-afef-6da0d657df79 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:02.669541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60409a9b-c603-40ac-afef-6da0d657df79 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:02.670182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60409a9b-c603-40ac-afef-6da0d657df79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:02.670498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60409a9b-c603-40ac-afef-6da0d657df79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:02.671884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60409a9b-c603-40ac-afef-6da0d657df79 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:02.685043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 563/1093] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:02.692198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60409a9b-c603-40ac-afef-6da0d657df79 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:02.779988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94f742-e2a4-49c2-8015-06079037a57b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:02.802568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60409a9b-c603-40ac-afef-6da0d657df79 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:02.802813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:02.818294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888abf8-e717-41bf-bafc-485a5b9773ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:02.824744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888abf8-e717-41bf-bafc-485a5b9773ee None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:02.824744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888abf8-e717-41bf-bafc-485a5b9773ee None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:02.824744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888abf8-e717-41bf-bafc-485a5b9773ee None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:02.824744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888abf8-e717-41bf-bafc-485a5b9773ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:02.824744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888abf8-e717-41bf-bafc-485a5b9773ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:02.824744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888abf8-e717-41bf-bafc-485a5b9773ee None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:02.826637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:02.834928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 531/1094] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:32:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:02.841753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888abf8-e717-41bf-bafc-485a5b9773ee None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:02.846624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579d540-0402-4db5-8749-d1297da67cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:02.847426 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579d540-0402-4db5-8749-d1297da67cbe None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:02.847727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579d540-0402-4db5-8749-d1297da67cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:02.848187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579d540-0402-4db5-8749-d1297da67cbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:02.856801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579d540-0402-4db5-8749-d1297da67cbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:02.858268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579d540-0402-4db5-8749-d1297da67cbe None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-f579d540-0402-4db5-8749-d1297da67cbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:32:02.870535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:02.872098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:02.873538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:02.873538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:02.873538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:32:02.873538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:32:02.873538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:32:02.873538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:32:02.873538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:32:02.873538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:32:02.873538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 23:32:02.873538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:32:02.873538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 564/1095] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Aug 29 23:32:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:32:02.877709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579d540-0402-4db5-8749-d1297da67cbe None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:02.952685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888abf8-e717-41bf-bafc-485a5b9773ee None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:02.988743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579d540-0402-4db5-8749-d1297da67cbe None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:02.991462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00497bd5-2253-42fe-8219-efa0a50521ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:02.992260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497bd5-2253-42fe-8219-efa0a50521ee None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:02.992668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497bd5-2253-42fe-8219-efa0a50521ee None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:02.993099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497bd5-2253-42fe-8219-efa0a50521ee None admin] PATH_INFO: `/v3/users/07fdcf436b6045379290daf80d0c3199` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:02.994526 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00497bd5-2253-42fe-8219-efa0a50521ee None admin] RBAC: Authorizing `identity:get_user(user_id=07fdcf436b6045379290daf80d0c3199)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:02.996137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00497bd5-2253-42fe-8219-efa0a50521ee None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:02.997792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 532/1096] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:02 2022] GET /identity/v3/users/07fdcf436b6045379290daf80d0c3199 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:03.003808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00497bd5-2253-42fe-8219-efa0a50521ee None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:03.026118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1bc5a-515e-421f-88dd-bd917abed8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:03.026779 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1bc5a-515e-421f-88dd-bd917abed8ca None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:03.027058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1bc5a-515e-421f-88dd-bd917abed8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:03.027365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1bc5a-515e-421f-88dd-bd917abed8ca None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:03.028101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1bc5a-515e-421f-88dd-bd917abed8ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:03.028406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1bc5a-515e-421f-88dd-bd917abed8ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:03.029868 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1bc5a-515e-421f-88dd-bd917abed8ca None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:03.050034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 565/1097] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Aug 29 23:32:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:03.058270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1bc5a-515e-421f-88dd-bd917abed8ca None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:03.062275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:03.112782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:03.122411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00497bd5-2253-42fe-8219-efa0a50521ee None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:03.173203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c6a1c-6542-4266-8fa9-ea90b0c35520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:03.173813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c6a1c-6542-4266-8fa9-ea90b0c35520 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:03.174090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c6a1c-6542-4266-8fa9-ea90b0c35520 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:03.174358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c6a1c-6542-4266-8fa9-ea90b0c35520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:03.192153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1bc5a-515e-421f-88dd-bd917abed8ca None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:03.229865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c31af-729b-40cc-b2ca-a5907634f971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:03.230508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c31af-729b-40cc-b2ca-a5907634f971 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:03.230790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c31af-729b-40cc-b2ca-a5907634f971 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:03.231066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c31af-729b-40cc-b2ca-a5907634f971 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:03.264396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c6a1c-6542-4266-8fa9-ea90b0c35520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:03.265859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c6a1c-6542-4266-8fa9-ea90b0c35520 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:03.317143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c31af-729b-40cc-b2ca-a5907634f971 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:03.317525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:03.318557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c31af-729b-40cc-b2ca-a5907634f971 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:03.367855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:03.391138 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 533/1098] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:32:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 23:32:03.397232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c6a1c-6542-4266-8fa9-ea90b0c35520 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:03.442920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 566/1099] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:32:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 23:32:03.449262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c31af-729b-40cc-b2ca-a5907634f971 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:03.514859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c6a1c-6542-4266-8fa9-ea90b0c35520 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:03.561733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf0c95-7e76-407a-a086-cf185fd8ecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:03.562575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf0c95-7e76-407a-a086-cf185fd8ecaf None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:03.562718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf0c95-7e76-407a-a086-cf185fd8ecaf None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:03.562933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf0c95-7e76-407a-a086-cf185fd8ecaf None admin] PATH_INFO: `/v3/projects/3ca9a513f6144fe6aa717b06782d83f2` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:03.564108 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf0c95-7e76-407a-a086-cf185fd8ecaf None admin] RBAC: Authorizing `identity:get_project(project_id=3ca9a513f6144fe6aa717b06782d83f2)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:03.565578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf0c95-7e76-407a-a086-cf185fd8ecaf None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:03.567360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 534/1100] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:32:03 2022] GET /identity/v3/projects/3ca9a513f6144fe6aa717b06782d83f2 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:03.570565 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c31af-729b-40cc-b2ca-a5907634f971 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:03.574206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf0c95-7e76-407a-a086-cf185fd8ecaf None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:03.595406 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:03.612360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d892de-340d-48cc-93e8-0bb18ba623be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:03.613229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d892de-340d-48cc-93e8-0bb18ba623be None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:03.613229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d892de-340d-48cc-93e8-0bb18ba623be None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:03.613547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d892de-340d-48cc-93e8-0bb18ba623be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:03.617294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:03.625579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d892de-340d-48cc-93e8-0bb18ba623be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:03.627090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d892de-340d-48cc-93e8-0bb18ba623be None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-24d892de-340d-48cc-93e8-0bb18ba623be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:32:03.643629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:03.644671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:32:03.645583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:32:03.645583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:32:03.645583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:32:03.645583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 567/1101] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:32:03.651266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d892de-340d-48cc-93e8-0bb18ba623be None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:03.694452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf0c95-7e76-407a-a086-cf185fd8ecaf None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:03.732161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6c849-e3a7-40a9-bb56-5ec4887056ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:03.732772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c849-e3a7-40a9-bb56-5ec4887056ee None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:03.733047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c849-e3a7-40a9-bb56-5ec4887056ee None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:03.733318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c849-e3a7-40a9-bb56-5ec4887056ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:03.734043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6c849-e3a7-40a9-bb56-5ec4887056ee None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=3ca9a513f6144fe6aa717b06782d83f2, user.id=07fdcf436b6045379290daf80d0c3199) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:03.734344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6c849-e3a7-40a9-bb56-5ec4887056ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:03.735962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6c849-e3a7-40a9-bb56-5ec4887056ee None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:03.742274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 535/1102] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:32:03 2022] GET /identity/v3/role_assignments?user.id=07fdcf436b6045379290daf80d0c3199&scope.project.id=3ca9a513f6144fe6aa717b06782d83f2&role.id=d86dba8efe20423abdde06031690beb8 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:03.773179 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d892de-340d-48cc-93e8-0bb18ba623be None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:03.816217 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0e0be-0fa0-4119-b8be-a3ea9e4490ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:03.816848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0e0be-0fa0-4119-b8be-a3ea9e4490ec None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:03.817712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0e0be-0fa0-4119-b8be-a3ea9e4490ec None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:03.817712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0e0be-0fa0-4119-b8be-a3ea9e4490ec None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:03.818060 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0e0be-0fa0-4119-b8be-a3ea9e4490ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:03.818365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0e0be-0fa0-4119-b8be-a3ea9e4490ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:03.819831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0e0be-0fa0-4119-b8be-a3ea9e4490ec None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:03.833899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 568/1103] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:03.840794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6c849-e3a7-40a9-bb56-5ec4887056ee None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:03.945477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:03.965669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6c849-e3a7-40a9-bb56-5ec4887056ee None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:04.010642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bf46a-e9b6-4b80-ac58-e3a1d807feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:04.011374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bf46a-e9b6-4b80-ac58-e3a1d807feea None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:04.011552 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bf46a-e9b6-4b80-ac58-e3a1d807feea None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:04.011817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bf46a-e9b6-4b80-ac58-e3a1d807feea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:04.019369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bf46a-e9b6-4b80-ac58-e3a1d807feea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:04.020826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bf46a-e9b6-4b80-ac58-e3a1d807feea None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-7b0bf46a-e9b6-4b80-ac58-e3a1d807feea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:32:04.033506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:04.035074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:04.036325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:32:04.036325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:32:04.036325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:32:04.036325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:32:04.036325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:32:04.036325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:32:04.036325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:32:04.036325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:32:04.036325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 536/1104] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:32:04.040668 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0e0be-0fa0-4119-b8be-a3ea9e4490ec None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:04.100702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:04.158266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0e0be-0fa0-4119-b8be-a3ea9e4490ec None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:04.193643 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e33fc-fdbc-4002-9814-bb6e5f8ddce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:04.194248 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e33fc-fdbc-4002-9814-bb6e5f8ddce6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:04.194554 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e33fc-fdbc-4002-9814-bb6e5f8ddce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:04.194821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e33fc-fdbc-4002-9814-bb6e5f8ddce6 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:04.195508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e33fc-fdbc-4002-9814-bb6e5f8ddce6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:04.195803 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e33fc-fdbc-4002-9814-bb6e5f8ddce6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:04.197294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e33fc-fdbc-4002-9814-bb6e5f8ddce6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:04.216122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 569/1105] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:32:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:04.222638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bf46a-e9b6-4b80-ac58-e3a1d807feea None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:04.357843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bf46a-e9b6-4b80-ac58-e3a1d807feea None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:04.399732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbf7cc-d4ac-4240-bf4a-b54cd2780060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:04.400498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbf7cc-d4ac-4240-bf4a-b54cd2780060 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:04.400905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbf7cc-d4ac-4240-bf4a-b54cd2780060 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:04.401187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbf7cc-d4ac-4240-bf4a-b54cd2780060 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:04.401936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbf7cc-d4ac-4240-bf4a-b54cd2780060 None admin] RBAC: Adding query filter params (role.id=c6a4683645ce4b68a14f2edeee414d92, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=e4a8261d92064b838fc6e8f415879566) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:04.402235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbf7cc-d4ac-4240-bf4a-b54cd2780060 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:04.403726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbf7cc-d4ac-4240-bf4a-b54cd2780060 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:04.410497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 537/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:32:04 2022] GET /identity/v3/role_assignments?user.id=e4a8261d92064b838fc6e8f415879566&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=c6a4683645ce4b68a14f2edeee414d92 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:05.780935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f634321-88a7-4f9d-bc6d-275331242e61 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:05.781461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f634321-88a7-4f9d-bc6d-275331242e61 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:05.781461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f634321-88a7-4f9d-bc6d-275331242e61 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:05.783038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 570/1107] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:32:05.788816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171bfb1-ba00-4f34-9073-f9ecfd4bb40a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:05.789308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171bfb1-ba00-4f34-9073-f9ecfd4bb40a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:05.789436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171bfb1-ba00-4f34-9073-f9ecfd4bb40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:05.803505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:05.810050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-6171bfb1-ba00-4f34-9073-f9ecfd4bb40a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:05.811309 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6171bfb1-ba00-4f34-9073-f9ecfd4bb40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:05.886442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 538/1108] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:32:06.163804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f634321-88a7-4f9d-bc6d-275331242e61 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:06.321288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f634321-88a7-4f9d-bc6d-275331242e61 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:06.340347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:06.383603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ed223-2f46-4517-8b7b-1a85ce0e0e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:06.384732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ed223-2f46-4517-8b7b-1a85ce0e0e69 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:06.385225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ed223-2f46-4517-8b7b-1a85ce0e0e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:06.385714 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ed223-2f46-4517-8b7b-1a85ce0e0e69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:06.387706 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ed223-2f46-4517-8b7b-1a85ce0e0e69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:06.390392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ed223-2f46-4517-8b7b-1a85ce0e0e69 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:06.393392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 571/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:32:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:06.399590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171bfb1-ba00-4f34-9073-f9ecfd4bb40a None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:06.514990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171bfb1-ba00-4f34-9073-f9ecfd4bb40a None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:06.557928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8798c-89d5-4ee6-be9e-4db1840f8529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:06.558582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8798c-89d5-4ee6-be9e-4db1840f8529 None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:06.558864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8798c-89d5-4ee6-be9e-4db1840f8529 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:06.559138 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8798c-89d5-4ee6-be9e-4db1840f8529 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:06.559836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8798c-89d5-4ee6-be9e-4db1840f8529 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:06.561336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8798c-89d5-4ee6-be9e-4db1840f8529 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:06.571434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:06.696714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 539/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:32:06 2022] POST /identity/v3/users => generated 286 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:32:08.768394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401eba6-bf27-45be-ac50-6dea9bcd34f8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:08.768946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401eba6-bf27-45be-ac50-6dea9bcd34f8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:08.768946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401eba6-bf27-45be-ac50-6dea9bcd34f8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:08.770214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 572/1111] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:32:08.776002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d439947-49c9-4367-a757-598a78f0c8b2 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:08.776501 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d439947-49c9-4367-a757-598a78f0c8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:08.776723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d439947-49c9-4367-a757-598a78f0c8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:08.796011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-2d439947-49c9-4367-a757-598a78f0c8b2 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:08.797220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2d439947-49c9-4367-a757-598a78f0c8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:08.869525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 540/1112] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:32:09.158984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401eba6-bf27-45be-ac50-6dea9bcd34f8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:09.271036 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401eba6-bf27-45be-ac50-6dea9bcd34f8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:09.307749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290fbcd-de53-49bd-8a78-36620a24afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:09.308685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290fbcd-de53-49bd-8a78-36620a24afb1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:09.309039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290fbcd-de53-49bd-8a78-36620a24afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:09.309314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290fbcd-de53-49bd-8a78-36620a24afb1 None admin] PATH_INFO: `/v3/roles/a38face2eeae40c28edda83a4d46614f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:09.310805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290fbcd-de53-49bd-8a78-36620a24afb1 None admin] RBAC: Authorizing `identity:get_role(role_id=a38face2eeae40c28edda83a4d46614f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:09.312184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290fbcd-de53-49bd-8a78-36620a24afb1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:09.313557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 573/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:09 2022] GET /identity/v3/roles/a38face2eeae40c28edda83a4d46614f => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:09.319812 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d439947-49c9-4367-a757-598a78f0c8b2 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:09.425258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d439947-49c9-4367-a757-598a78f0c8b2 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:09.450979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:09.463663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4ec5e-6d0a-4996-a1d4-6a9d6458c132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:09.464297 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ec5e-6d0a-4996-a1d4-6a9d6458c132 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:09.464577 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ec5e-6d0a-4996-a1d4-6a9d6458c132 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:09.464852 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ec5e-6d0a-4996-a1d4-6a9d6458c132 None admin] PATH_INFO: `/v3/users/180aa99bf9d24fa9a247909efd3e6480` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:09.480939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4ec5e-6d0a-4996-a1d4-6a9d6458c132 None admin] RBAC: Authorizing `identity:get_user(user_id=180aa99bf9d24fa9a247909efd3e6480)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:09.482573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4ec5e-6d0a-4996-a1d4-6a9d6458c132 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:09.484848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 541/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:09 2022] GET /identity/v3/users/180aa99bf9d24fa9a247909efd3e6480 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:09.490483 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290fbcd-de53-49bd-8a78-36620a24afb1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:09.541372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:09.598506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290fbcd-de53-49bd-8a78-36620a24afb1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:09.635090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f961051-4c95-4d53-a5f2-c9cfd662d5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:09.635702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f961051-4c95-4d53-a5f2-c9cfd662d5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:09.635979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f961051-4c95-4d53-a5f2-c9cfd662d5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:09.636251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f961051-4c95-4d53-a5f2-c9cfd662d5c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:09.710663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f961051-4c95-4d53-a5f2-c9cfd662d5c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:09.712039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f961051-4c95-4d53-a5f2-c9cfd662d5c5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:09.762021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:09.819905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 574/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:32:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:32:09.826871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4ec5e-6d0a-4996-a1d4-6a9d6458c132 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:09.961720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4ec5e-6d0a-4996-a1d4-6a9d6458c132 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:10.000556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c81ec-d688-4009-a15f-f46aa3fc10c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:10.001189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c81ec-d688-4009-a15f-f46aa3fc10c8 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:10.001472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c81ec-d688-4009-a15f-f46aa3fc10c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:10.001795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c81ec-d688-4009-a15f-f46aa3fc10c8 None admin] PATH_INFO: `/v3/projects/3ca9a513f6144fe6aa717b06782d83f2` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:10.003173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c81ec-d688-4009-a15f-f46aa3fc10c8 None admin] RBAC: Authorizing `identity:get_project(project_id=3ca9a513f6144fe6aa717b06782d83f2)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:10.004833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c81ec-d688-4009-a15f-f46aa3fc10c8 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:10.007057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 542/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:32:09 2022] GET /identity/v3/projects/3ca9a513f6144fe6aa717b06782d83f2 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:10.015097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f961051-4c95-4d53-a5f2-c9cfd662d5c5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:10.119154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f961051-4c95-4d53-a5f2-c9cfd662d5c5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:10.154708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb0f4e-b1a4-4042-9367-f578a94dfd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:10.155455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb0f4e-b1a4-4042-9367-f578a94dfd40 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:10.155735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb0f4e-b1a4-4042-9367-f578a94dfd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:10.156021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb0f4e-b1a4-4042-9367-f578a94dfd40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:10.157115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb0f4e-b1a4-4042-9367-f578a94dfd40 None admin] RBAC: Adding query filter params (role.id=a38face2eeae40c28edda83a4d46614f, scope.project.id=3ca9a513f6144fe6aa717b06782d83f2, user.id=180aa99bf9d24fa9a247909efd3e6480) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:10.157526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb0f4e-b1a4-4042-9367-f578a94dfd40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:10.159694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb0f4e-b1a4-4042-9367-f578a94dfd40 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:10.166668 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 575/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:32:10 2022] GET /identity/v3/role_assignments?user.id=180aa99bf9d24fa9a247909efd3e6480&scope.project.id=3ca9a513f6144fe6aa717b06782d83f2&role.id=a38face2eeae40c28edda83a4d46614f => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:12.126848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2608d0-910f-4bfe-b159-e87e1bc86695 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:12.127706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2608d0-910f-4bfe-b159-e87e1bc86695 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:12.127706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2608d0-910f-4bfe-b159-e87e1bc86695 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:12.129212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 543/1118] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:32:12.136684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b7c35-ed18-4666-85a5-bef312b0e241 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:12.137222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b7c35-ed18-4666-85a5-bef312b0e241 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:12.137904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b7c35-ed18-4666-85a5-bef312b0e241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:12.166409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b80b7c35-ed18-4666-85a5-bef312b0e241 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:12.168166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b80b7c35-ed18-4666-85a5-bef312b0e241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:12.171542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:12.281957 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 576/1119] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:32:12.563531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2608d0-910f-4bfe-b159-e87e1bc86695 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:12.617404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:12.673823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2608d0-910f-4bfe-b159-e87e1bc86695 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:12.714593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd168d6d-0f19-4c90-b1df-b07bb2010904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:12.715250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd168d6d-0f19-4c90-b1df-b07bb2010904 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:12.715529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd168d6d-0f19-4c90-b1df-b07bb2010904 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:12.715796 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd168d6d-0f19-4c90-b1df-b07bb2010904 None admin] PATH_INFO: `/v3/roles/a38face2eeae40c28edda83a4d46614f` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:12.717498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd168d6d-0f19-4c90-b1df-b07bb2010904 None admin] RBAC: Authorizing `identity:get_role(role_id=a38face2eeae40c28edda83a4d46614f)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:12.719050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd168d6d-0f19-4c90-b1df-b07bb2010904 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:12.720488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 544/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:12 2022] GET /identity/v3/roles/a38face2eeae40c28edda83a4d46614f => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:12.726936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b7c35-ed18-4666-85a5-bef312b0e241 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:12.874093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b7c35-ed18-4666-85a5-bef312b0e241 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:12.910523 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ef293-f8f8-409d-b0b3-9bbea18506b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:12.911169 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ef293-f8f8-409d-b0b3-9bbea18506b9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:12.911429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ef293-f8f8-409d-b0b3-9bbea18506b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:12.911697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ef293-f8f8-409d-b0b3-9bbea18506b9 None admin] PATH_INFO: `/v3/users/180aa99bf9d24fa9a247909efd3e6480` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:12.912864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ef293-f8f8-409d-b0b3-9bbea18506b9 None admin] RBAC: Authorizing `identity:get_user(user_id=180aa99bf9d24fa9a247909efd3e6480)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:12.914321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ef293-f8f8-409d-b0b3-9bbea18506b9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:12.916157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 577/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:12 2022] GET /identity/v3/users/180aa99bf9d24fa9a247909efd3e6480 => generated 286 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:12.922356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd168d6d-0f19-4c90-b1df-b07bb2010904 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:13.034252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd168d6d-0f19-4c90-b1df-b07bb2010904 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:13.052986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:13.072834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1df336-8867-4330-b800-d26c91262b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:13.073486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1df336-8867-4330-b800-d26c91262b35 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:13.073763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1df336-8867-4330-b800-d26c91262b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:13.074033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1df336-8867-4330-b800-d26c91262b35 None admin] PATH_INFO: `/v3/projects/3ca9a513f6144fe6aa717b06782d83f2` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:13.075288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1df336-8867-4330-b800-d26c91262b35 None admin] RBAC: Authorizing `identity:get_project(project_id=3ca9a513f6144fe6aa717b06782d83f2)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:13.076731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1df336-8867-4330-b800-d26c91262b35 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:13.078702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 545/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:32:12 2022] GET /identity/v3/projects/3ca9a513f6144fe6aa717b06782d83f2 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:13.084662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ef293-f8f8-409d-b0b3-9bbea18506b9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:13.094590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:13.197945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ef293-f8f8-409d-b0b3-9bbea18506b9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:13.236272 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a4dbf-535b-4d43-9157-3c506a0ba8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:13.236927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a4dbf-535b-4d43-9157-3c506a0ba8bf None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:13.237201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a4dbf-535b-4d43-9157-3c506a0ba8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:13.237471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a4dbf-535b-4d43-9157-3c506a0ba8bf None admin] PATH_INFO: `/v3/projects/3ca9a513f6144fe6aa717b06782d83f2/users/180aa99bf9d24fa9a247909efd3e6480/roles/a38face2eeae40c28edda83a4d46614f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:13.239649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a4dbf-535b-4d43-9157-3c506a0ba8bf None admin] RBAC: Authorizing `identity:create_grant(project_id=3ca9a513f6144fe6aa717b06782d83f2, user_id=180aa99bf9d24fa9a247909efd3e6480, role_id=a38face2eeae40c28edda83a4d46614f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:13.241092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a4dbf-535b-4d43-9157-3c506a0ba8bf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:13.302076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 578/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:32:13 2022] PUT /identity/v3/projects/3ca9a513f6144fe6aa717b06782d83f2/users/180aa99bf9d24fa9a247909efd3e6480/roles/a38face2eeae40c28edda83a4d46614f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:32:15.304429 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243c38b-7b09-445d-848f-7c1cf04a20de None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:15.305059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243c38b-7b09-445d-848f-7c1cf04a20de None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:15.305059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243c38b-7b09-445d-848f-7c1cf04a20de None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:15.307061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 546/1124] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:32:15.313376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582b79b-78b6-4cd0-8c83-20cb29e4f4ca None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:15.313930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582b79b-78b6-4cd0-8c83-20cb29e4f4ca None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:15.313930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582b79b-78b6-4cd0-8c83-20cb29e4f4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:15.336065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3582b79b-78b6-4cd0-8c83-20cb29e4f4ca None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:15.337843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3582b79b-78b6-4cd0-8c83-20cb29e4f4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:15.403999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:15.411961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 579/1125] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:32:15.690130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243c38b-7b09-445d-848f-7c1cf04a20de None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:15.806585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243c38b-7b09-445d-848f-7c1cf04a20de None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:15.844643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d3355-1390-46c3-9663-694a0289660c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:15.845281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d3355-1390-46c3-9663-694a0289660c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:15.845561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d3355-1390-46c3-9663-694a0289660c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:15.845833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d3355-1390-46c3-9663-694a0289660c None admin] PATH_INFO: `/v3/roles/a38face2eeae40c28edda83a4d46614f` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:15.847284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d3355-1390-46c3-9663-694a0289660c None admin] RBAC: Authorizing `identity:get_role(role_id=a38face2eeae40c28edda83a4d46614f)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:15.848656 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d3355-1390-46c3-9663-694a0289660c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:15.850030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 547/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:15 2022] GET /identity/v3/roles/a38face2eeae40c28edda83a4d46614f => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:15.856957 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582b79b-78b6-4cd0-8c83-20cb29e4f4ca None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:15.966035 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582b79b-78b6-4cd0-8c83-20cb29e4f4ca None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:16.005591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67614d03-28c0-45c0-a5a0-aa9bc4d38fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:16.006235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67614d03-28c0-45c0-a5a0-aa9bc4d38fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:16.006541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67614d03-28c0-45c0-a5a0-aa9bc4d38fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:16.006841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67614d03-28c0-45c0-a5a0-aa9bc4d38fe3 None admin] PATH_INFO: `/v3/users/180aa99bf9d24fa9a247909efd3e6480` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:16.007996 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67614d03-28c0-45c0-a5a0-aa9bc4d38fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=180aa99bf9d24fa9a247909efd3e6480)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:16.009608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67614d03-28c0-45c0-a5a0-aa9bc4d38fe3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:16.011280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 580/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:15 2022] GET /identity/v3/users/180aa99bf9d24fa9a247909efd3e6480 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:16.017653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d3355-1390-46c3-9663-694a0289660c None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:16.075741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:16.125041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d3355-1390-46c3-9663-694a0289660c None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:16.161451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812da66c-1e67-43dd-b0c7-ea7a4c75fae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:16.162056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812da66c-1e67-43dd-b0c7-ea7a4c75fae9 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:16.162333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812da66c-1e67-43dd-b0c7-ea7a4c75fae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:16.162633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812da66c-1e67-43dd-b0c7-ea7a4c75fae9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:16.250966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812da66c-1e67-43dd-b0c7-ea7a4c75fae9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:16.252326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812da66c-1e67-43dd-b0c7-ea7a4c75fae9 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:16.314354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:16.365516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 548/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:32:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:32:16.374484 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67614d03-28c0-45c0-a5a0-aa9bc4d38fe3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:16.474893 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:16.545516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67614d03-28c0-45c0-a5a0-aa9bc4d38fe3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:16.594879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609da18e-a180-4b92-82c7-1cc8a06c729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:16.595515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609da18e-a180-4b92-82c7-1cc8a06c729a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:16.595790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609da18e-a180-4b92-82c7-1cc8a06c729a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:16.596057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609da18e-a180-4b92-82c7-1cc8a06c729a None admin] PATH_INFO: `/v3/projects/3ca9a513f6144fe6aa717b06782d83f2` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:16.597275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609da18e-a180-4b92-82c7-1cc8a06c729a None admin] RBAC: Authorizing `identity:get_project(project_id=3ca9a513f6144fe6aa717b06782d83f2)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:16.598777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609da18e-a180-4b92-82c7-1cc8a06c729a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:16.600510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 581/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:32:16 2022] GET /identity/v3/projects/3ca9a513f6144fe6aa717b06782d83f2 => generated 311 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:16.609916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812da66c-1e67-43dd-b0c7-ea7a4c75fae9 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:16.732917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812da66c-1e67-43dd-b0c7-ea7a4c75fae9 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:16.775601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d1cb1-0ff8-498b-9878-db997ee616f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:16.776204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d1cb1-0ff8-498b-9878-db997ee616f5 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:16.776479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d1cb1-0ff8-498b-9878-db997ee616f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:16.776746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d1cb1-0ff8-498b-9878-db997ee616f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:16.777476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d1cb1-0ff8-498b-9878-db997ee616f5 None admin] RBAC: Adding query filter params (role.id=a38face2eeae40c28edda83a4d46614f, scope.project.id=3ca9a513f6144fe6aa717b06782d83f2, user.id=180aa99bf9d24fa9a247909efd3e6480) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:16.777779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d1cb1-0ff8-498b-9878-db997ee616f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:16.779251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d1cb1-0ff8-498b-9878-db997ee616f5 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:16.787945 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 549/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:32:16 2022] GET /identity/v3/role_assignments?user.id=180aa99bf9d24fa9a247909efd3e6480&scope.project.id=3ca9a513f6144fe6aa717b06782d83f2&role.id=a38face2eeae40c28edda83a4d46614f => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:18.836916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76348901-cc30-48d9-9b29-2d0746558370 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:18.837443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76348901-cc30-48d9-9b29-2d0746558370 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:18.837443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76348901-cc30-48d9-9b29-2d0746558370 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:18.838888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 582/1131] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:32:18.847528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837552fe-cb87-46d8-a59f-4a1b3e1cfb0d None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:18.848084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837552fe-cb87-46d8-a59f-4a1b3e1cfb0d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:18.848084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837552fe-cb87-46d8-a59f-4a1b3e1cfb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:18.868428 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-837552fe-cb87-46d8-a59f-4a1b3e1cfb0d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:18.869695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-837552fe-cb87-46d8-a59f-4a1b3e1cfb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:18.883025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:18.949397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 550/1132] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:32:19.232184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76348901-cc30-48d9-9b29-2d0746558370 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:19.350764 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76348901-cc30-48d9-9b29-2d0746558370 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:19.382914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:19.389057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf675a-c62c-4592-9f34-795f43004164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:19.389701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf675a-c62c-4592-9f34-795f43004164 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:19.389987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf675a-c62c-4592-9f34-795f43004164 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:19.390261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf675a-c62c-4592-9f34-795f43004164 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:19.391467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf675a-c62c-4592-9f34-795f43004164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:19.404479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf675a-c62c-4592-9f34-795f43004164 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:19.404479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 583/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:32:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:19.404816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837552fe-cb87-46d8-a59f-4a1b3e1cfb0d None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:19.508883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837552fe-cb87-46d8-a59f-4a1b3e1cfb0d None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:19.546595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cf245-6121-486c-a018-6b723d49da2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:19.547202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf245-6121-486c-a018-6b723d49da2c None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:19.547478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf245-6121-486c-a018-6b723d49da2c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:19.547746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf245-6121-486c-a018-6b723d49da2c None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:19.548414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92cf245-6121-486c-a018-6b723d49da2c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:19.549840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92cf245-6121-486c-a018-6b723d49da2c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:19.639037 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:19.646047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 551/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:32:19 2022] POST /identity/v3/projects => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:32:21.684800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e9d24-b9d0-46d1-9c7b-58f9a1011e8d None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:21.685366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e9d24-b9d0-46d1-9c7b-58f9a1011e8d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:21.685366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e9d24-b9d0-46d1-9c7b-58f9a1011e8d None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:21.686701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 584/1135] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:32:21.692898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d18e3-afa7-43a4-8974-e93cd1b5597a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:21.693504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d18e3-afa7-43a4-8974-e93cd1b5597a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:21.693504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d18e3-afa7-43a4-8974-e93cd1b5597a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:21.714103 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-c47d18e3-afa7-43a4-8974-e93cd1b5597a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:21.715439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c47d18e3-afa7-43a4-8974-e93cd1b5597a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:21.812708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 552/1136] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:32:22.092759 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e9d24-b9d0-46d1-9c7b-58f9a1011e8d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:22.203513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e9d24-b9d0-46d1-9c7b-58f9a1011e8d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:22.240579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77031377-6b64-4a47-ad76-1c3d6f53f81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:22.241285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77031377-6b64-4a47-ad76-1c3d6f53f81a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:22.241609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77031377-6b64-4a47-ad76-1c3d6f53f81a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:22.241928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77031377-6b64-4a47-ad76-1c3d6f53f81a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:22.243515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77031377-6b64-4a47-ad76-1c3d6f53f81a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:22.245068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77031377-6b64-4a47-ad76-1c3d6f53f81a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:22.246743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 585/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:32:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:22.253371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d18e3-afa7-43a4-8974-e93cd1b5597a None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:22.419432 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d18e3-afa7-43a4-8974-e93cd1b5597a None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:22.450641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:22.459309 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83966c-497a-44c4-9076-7ad40cf57fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:22.459966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83966c-497a-44c4-9076-7ad40cf57fb2 None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:22.475425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83966c-497a-44c4-9076-7ad40cf57fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:22.475425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83966c-497a-44c4-9076-7ad40cf57fb2 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:22.475425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83966c-497a-44c4-9076-7ad40cf57fb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:22.475425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83966c-497a-44c4-9076-7ad40cf57fb2 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:22.570301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 553/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:32:22 2022] POST /identity/v3/users => generated 286 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:32:25.231374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a287647-a037-4b7e-b763-2291d7bf5b5d None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:25.231997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a287647-a037-4b7e-b763-2291d7bf5b5d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:25.231997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a287647-a037-4b7e-b763-2291d7bf5b5d None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:25.233401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 586/1139] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:32:25.239283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bdf6d-2b70-4154-97c1-f7696587053d None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:25.239766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bdf6d-2b70-4154-97c1-f7696587053d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:25.239924 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bdf6d-2b70-4154-97c1-f7696587053d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:25.261278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-0a5bdf6d-2b70-4154-97c1-f7696587053d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:25.262942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-0a5bdf6d-2b70-4154-97c1-f7696587053d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:25.355086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 554/1140] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:32:25.642042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a287647-a037-4b7e-b763-2291d7bf5b5d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:25.715719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:25.752270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a287647-a037-4b7e-b763-2291d7bf5b5d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:25.790118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681070c-e1ea-40fc-8c9e-ae410e3dbb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:25.790860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681070c-e1ea-40fc-8c9e-ae410e3dbb49 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:25.791181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681070c-e1ea-40fc-8c9e-ae410e3dbb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:25.791492 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681070c-e1ea-40fc-8c9e-ae410e3dbb49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:25.803118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681070c-e1ea-40fc-8c9e-ae410e3dbb49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1681070c-e1ea-40fc-8c9e-ae410e3dbb49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:32:25.809299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:25.811256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:32:25.812966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 587/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:32:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:32:25.817964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bdf6d-2b70-4154-97c1-f7696587053d None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:25.927189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bdf6d-2b70-4154-97c1-f7696587053d None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:25.943484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:25.966291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44afb34-975f-4885-a829-acd2015efa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:25.967050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44afb34-975f-4885-a829-acd2015efa99 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:25.967432 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44afb34-975f-4885-a829-acd2015efa99 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:25.967796 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44afb34-975f-4885-a829-acd2015efa99 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:25.968558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44afb34-975f-4885-a829-acd2015efa99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:25.968951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44afb34-975f-4885-a829-acd2015efa99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:25.970458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44afb34-975f-4885-a829-acd2015efa99 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:25.981107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 555/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:32:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:25.987305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681070c-e1ea-40fc-8c9e-ae410e3dbb49 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:26.097459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681070c-e1ea-40fc-8c9e-ae410e3dbb49 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:26.128690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:26.138728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f971d76-8b7c-43fc-a2ba-6b49234fdc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:26.139379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f971d76-8b7c-43fc-a2ba-6b49234fdc30 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:26.139653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f971d76-8b7c-43fc-a2ba-6b49234fdc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:26.139922 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f971d76-8b7c-43fc-a2ba-6b49234fdc30 None admin] PATH_INFO: `/v3/users/adb02e58653a472baf7aa3efd9dbda3a` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:26.156652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f971d76-8b7c-43fc-a2ba-6b49234fdc30 None admin] RBAC: Authorizing `identity:get_user(user_id=adb02e58653a472baf7aa3efd9dbda3a)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:26.158464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f971d76-8b7c-43fc-a2ba-6b49234fdc30 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:26.160606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 588/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:25 2022] GET /identity/v3/users/adb02e58653a472baf7aa3efd9dbda3a => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:26.168057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44afb34-975f-4885-a829-acd2015efa99 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:26.281958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44afb34-975f-4885-a829-acd2015efa99 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:26.318655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0560a59-5c2b-485f-8c08-996ba467e8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:26.319271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0560a59-5c2b-485f-8c08-996ba467e8fa None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:26.319545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0560a59-5c2b-485f-8c08-996ba467e8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:26.319813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0560a59-5c2b-485f-8c08-996ba467e8fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:26.374474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:26.399938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0560a59-5c2b-485f-8c08-996ba467e8fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:26.401710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0560a59-5c2b-485f-8c08-996ba467e8fa None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:26.511414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 556/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:32:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 23:32:26.517954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f971d76-8b7c-43fc-a2ba-6b49234fdc30 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:26.627390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f971d76-8b7c-43fc-a2ba-6b49234fdc30 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:26.664997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657b31c-bc16-43e4-a526-fe03f489950b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:26.665629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b31c-bc16-43e4-a526-fe03f489950b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:26.665908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b31c-bc16-43e4-a526-fe03f489950b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:26.666183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b31c-bc16-43e4-a526-fe03f489950b None admin] PATH_INFO: `/v3/projects/0944f2bfa56245bfb81610bb5b1bc66c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:26.667382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657b31c-bc16-43e4-a526-fe03f489950b None admin] RBAC: Authorizing `identity:get_project(project_id=0944f2bfa56245bfb81610bb5b1bc66c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:26.668849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657b31c-bc16-43e4-a526-fe03f489950b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:26.670568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 589/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:32:26 2022] GET /identity/v3/projects/0944f2bfa56245bfb81610bb5b1bc66c => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:26.682863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0560a59-5c2b-485f-8c08-996ba467e8fa None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:26.819524 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:26.846985 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0560a59-5c2b-485f-8c08-996ba467e8fa None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:26.883562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11965521-ae02-43a7-91d6-e6115b2f3ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:26.884250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11965521-ae02-43a7-91d6-e6115b2f3ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:26.884531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11965521-ae02-43a7-91d6-e6115b2f3ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:26.884806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11965521-ae02-43a7-91d6-e6115b2f3ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:26.885536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11965521-ae02-43a7-91d6-e6115b2f3ef5 None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=0944f2bfa56245bfb81610bb5b1bc66c, user.id=adb02e58653a472baf7aa3efd9dbda3a) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:26.885848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11965521-ae02-43a7-91d6-e6115b2f3ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:26.887589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11965521-ae02-43a7-91d6-e6115b2f3ef5 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:26.893679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 557/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:32:26 2022] GET /identity/v3/role_assignments?user.id=adb02e58653a472baf7aa3efd9dbda3a&scope.project.id=0944f2bfa56245bfb81610bb5b1bc66c&role.id=d86dba8efe20423abdde06031690beb8 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:28.918400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebc555-c59d-46f4-86e5-ed962f6b66d9 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:28.918992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebc555-c59d-46f4-86e5-ed962f6b66d9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:28.919089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebc555-c59d-46f4-86e5-ed962f6b66d9 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:28.920760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 590/1147] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:32:28.926548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325986d0-dac5-4bbe-bf8c-49a692f9babe None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:28.927033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325986d0-dac5-4bbe-bf8c-49a692f9babe None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:28.927179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325986d0-dac5-4bbe-bf8c-49a692f9babe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:28.949052 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-325986d0-dac5-4bbe-bf8c-49a692f9babe None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:28.950260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-325986d0-dac5-4bbe-bf8c-49a692f9babe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:29.026982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 558/1148] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:32:29.305046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebc555-c59d-46f4-86e5-ed962f6b66d9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:29.383629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:29.433343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebc555-c59d-46f4-86e5-ed962f6b66d9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:29.468319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1505-fe5c-44fb-96ee-f1b7f1238352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:29.468945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1505-fe5c-44fb-96ee-f1b7f1238352 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:29.469226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1505-fe5c-44fb-96ee-f1b7f1238352 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:29.469498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1505-fe5c-44fb-96ee-f1b7f1238352 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:29.481116 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc1505-fe5c-44fb-96ee-f1b7f1238352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-2ffc1505-fe5c-44fb-96ee-f1b7f1238352 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:32:29.485972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:29.487123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:32:29.487986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 591/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:32:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:32:29.493627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325986d0-dac5-4bbe-bf8c-49a692f9babe None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:29.565254 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:29.597931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325986d0-dac5-4bbe-bf8c-49a692f9babe None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:29.633180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd149436-9919-48b4-8b0b-5f72084368ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:29.633786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd149436-9919-48b4-8b0b-5f72084368ef None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:29.634062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd149436-9919-48b4-8b0b-5f72084368ef None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:29.634332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd149436-9919-48b4-8b0b-5f72084368ef None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:29.635058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd149436-9919-48b4-8b0b-5f72084368ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:29.635359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd149436-9919-48b4-8b0b-5f72084368ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:29.637497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd149436-9919-48b4-8b0b-5f72084368ef None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:29.646378 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 559/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:32:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:29.653476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1505-fe5c-44fb-96ee-f1b7f1238352 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:29.766075 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1505-fe5c-44fb-96ee-f1b7f1238352 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:29.785109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:29.810154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11752ed-9f4c-4819-bf91-8dff2d1efa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:29.810926 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11752ed-9f4c-4819-bf91-8dff2d1efa41 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:29.811254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11752ed-9f4c-4819-bf91-8dff2d1efa41 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:29.811624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11752ed-9f4c-4819-bf91-8dff2d1efa41 None admin] PATH_INFO: `/v3/users/adb02e58653a472baf7aa3efd9dbda3a` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:29.813313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11752ed-9f4c-4819-bf91-8dff2d1efa41 None admin] RBAC: Authorizing `identity:get_user(user_id=adb02e58653a472baf7aa3efd9dbda3a)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:29.815384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11752ed-9f4c-4819-bf91-8dff2d1efa41 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:29.817407 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 592/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:29 2022] GET /identity/v3/users/adb02e58653a472baf7aa3efd9dbda3a => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:29.824365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd149436-9919-48b4-8b0b-5f72084368ef None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:29.943196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd149436-9919-48b4-8b0b-5f72084368ef None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:29.978173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:29.983632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fef36-eb8e-46bf-9647-3940ba4f907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:29.984293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fef36-eb8e-46bf-9647-3940ba4f907e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:29.984571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fef36-eb8e-46bf-9647-3940ba4f907e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:29.984845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fef36-eb8e-46bf-9647-3940ba4f907e None admin] PATH_INFO: `/v3/projects/0944f2bfa56245bfb81610bb5b1bc66c` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:29.986149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fef36-eb8e-46bf-9647-3940ba4f907e None admin] RBAC: Authorizing `identity:get_project(project_id=0944f2bfa56245bfb81610bb5b1bc66c)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:29.987675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fef36-eb8e-46bf-9647-3940ba4f907e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:29.989457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 560/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:32:29 2022] GET /identity/v3/projects/0944f2bfa56245bfb81610bb5b1bc66c => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:29.996786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11752ed-9f4c-4819-bf91-8dff2d1efa41 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:30.110682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11752ed-9f4c-4819-bf91-8dff2d1efa41 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:30.149475 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0d6bb-49ef-4725-bd94-4d6b9ccfa5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:30.150145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0d6bb-49ef-4725-bd94-4d6b9ccfa5e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:30.150453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0d6bb-49ef-4725-bd94-4d6b9ccfa5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:30.150727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0d6bb-49ef-4725-bd94-4d6b9ccfa5e0 None admin] PATH_INFO: `/v3/projects/0944f2bfa56245bfb81610bb5b1bc66c/users/adb02e58653a472baf7aa3efd9dbda3a/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:30.152680 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0d6bb-49ef-4725-bd94-4d6b9ccfa5e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0944f2bfa56245bfb81610bb5b1bc66c, user_id=adb02e58653a472baf7aa3efd9dbda3a, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:30.154139 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0d6bb-49ef-4725-bd94-4d6b9ccfa5e0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:30.239635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 593/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:32:29 2022] PUT /identity/v3/projects/0944f2bfa56245bfb81610bb5b1bc66c/users/adb02e58653a472baf7aa3efd9dbda3a/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:32:32.289663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d68ec-0250-4e37-8774-71bc11391fa8 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:32.290285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d68ec-0250-4e37-8774-71bc11391fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:32.290285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d68ec-0250-4e37-8774-71bc11391fa8 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:32.291900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 561/1154] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:32:32.298069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd9056-4d82-489c-9a03-8d94e5d04825 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:32.298749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd9056-4d82-489c-9a03-8d94e5d04825 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:32.298749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd9056-4d82-489c-9a03-8d94e5d04825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:32.318052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5acd9056-4d82-489c-9a03-8d94e5d04825 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:32.319288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5acd9056-4d82-489c-9a03-8d94e5d04825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:32.367522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:32.404317 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 594/1155] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:32:32.691189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d68ec-0250-4e37-8774-71bc11391fa8 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:32.802510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d68ec-0250-4e37-8774-71bc11391fa8 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:32.839652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae676c-8528-4933-91fd-bc7c2745da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:32.840293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae676c-8528-4933-91fd-bc7c2745da49 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:32.840569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae676c-8528-4933-91fd-bc7c2745da49 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:32.840836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae676c-8528-4933-91fd-bc7c2745da49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:32.853654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae676c-8528-4933-91fd-bc7c2745da49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-70ae676c-8528-4933-91fd-bc7c2745da49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:32:32.858514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:32.860174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:32:32.861743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 562/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:32:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:32:32.866239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd9056-4d82-489c-9a03-8d94e5d04825 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:32.976790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd9056-4d82-489c-9a03-8d94e5d04825 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:33.035040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17c008-dd08-485f-9733-98e83503f646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:33.036061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17c008-dd08-485f-9733-98e83503f646 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:33.036557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17c008-dd08-485f-9733-98e83503f646 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:33.037061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17c008-dd08-485f-9733-98e83503f646 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:33.038824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17c008-dd08-485f-9733-98e83503f646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:33.039403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17c008-dd08-485f-9733-98e83503f646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:33.041988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17c008-dd08-485f-9733-98e83503f646 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:33.058010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 595/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:32:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:33.064154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae676c-8528-4933-91fd-bc7c2745da49 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:33.129362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:33.174163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae676c-8528-4933-91fd-bc7c2745da49 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:33.213252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301925e-ba39-4890-b829-07126889de82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:33.213955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301925e-ba39-4890-b829-07126889de82 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:33.214296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301925e-ba39-4890-b829-07126889de82 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:33.214653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301925e-ba39-4890-b829-07126889de82 None admin] PATH_INFO: `/v3/users/adb02e58653a472baf7aa3efd9dbda3a` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:33.215939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301925e-ba39-4890-b829-07126889de82 None admin] RBAC: Authorizing `identity:get_user(user_id=adb02e58653a472baf7aa3efd9dbda3a)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:33.217462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301925e-ba39-4890-b829-07126889de82 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:33.219222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 563/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:33 2022] GET /identity/v3/users/adb02e58653a472baf7aa3efd9dbda3a => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:33.225494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17c008-dd08-485f-9733-98e83503f646 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:33.264587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:33.365619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17c008-dd08-485f-9733-98e83503f646 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:33.404217 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec654b-b19f-46d0-be43-25d45793508a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:33.404826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec654b-b19f-46d0-be43-25d45793508a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:33.405104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec654b-b19f-46d0-be43-25d45793508a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:33.405375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec654b-b19f-46d0-be43-25d45793508a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:33.493267 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec654b-b19f-46d0-be43-25d45793508a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:33.494806 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec654b-b19f-46d0-be43-25d45793508a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:33.542333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:33.630018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 596/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:32:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 23:32:33.637829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301925e-ba39-4890-b829-07126889de82 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:33.753619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301925e-ba39-4890-b829-07126889de82 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:33.783588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:33.798221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f3900-8e08-4fe5-bb24-494d9d3df52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:33.798221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f3900-8e08-4fe5-bb24-494d9d3df52b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:33.798221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f3900-8e08-4fe5-bb24-494d9d3df52b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:33.798221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f3900-8e08-4fe5-bb24-494d9d3df52b None admin] PATH_INFO: `/v3/projects/0944f2bfa56245bfb81610bb5b1bc66c` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:33.798783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f3900-8e08-4fe5-bb24-494d9d3df52b None admin] RBAC: Authorizing `identity:get_project(project_id=0944f2bfa56245bfb81610bb5b1bc66c)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:33.798783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f3900-8e08-4fe5-bb24-494d9d3df52b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:33.798783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 564/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:32:33 2022] GET /identity/v3/projects/0944f2bfa56245bfb81610bb5b1bc66c => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:33.805399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec654b-b19f-46d0-be43-25d45793508a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:33.919200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec654b-b19f-46d0-be43-25d45793508a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:33.962567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0554570-4eee-4b69-8e90-f6507b711ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:33.963254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0554570-4eee-4b69-8e90-f6507b711ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:33.963582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0554570-4eee-4b69-8e90-f6507b711ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:33.963902 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0554570-4eee-4b69-8e90-f6507b711ec3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:33.964722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0554570-4eee-4b69-8e90-f6507b711ec3 None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=0944f2bfa56245bfb81610bb5b1bc66c, user.id=adb02e58653a472baf7aa3efd9dbda3a) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:33.965090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0554570-4eee-4b69-8e90-f6507b711ec3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:33.966712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0554570-4eee-4b69-8e90-f6507b711ec3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:33.968896 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:33.975275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 597/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:32:33 2022] GET /identity/v3/role_assignments?user.id=adb02e58653a472baf7aa3efd9dbda3a&scope.project.id=0944f2bfa56245bfb81610bb5b1bc66c&role.id=d86dba8efe20423abdde06031690beb8 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:36.006802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ebf2-e69d-433c-9b5d-e31bb5060ea5 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:36.007457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ebf2-e69d-433c-9b5d-e31bb5060ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:36.007457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ebf2-e69d-433c-9b5d-e31bb5060ea5 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:36.008971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 565/1162] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:32:36.015139 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b639d-2da5-4a88-b5c1-734af8ad41c6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:36.015742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b639d-2da5-4a88-b5c1-734af8ad41c6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:36.015742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b639d-2da5-4a88-b5c1-734af8ad41c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:36.036369 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-450b639d-2da5-4a88-b5c1-734af8ad41c6 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:36.037644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-450b639d-2da5-4a88-b5c1-734af8ad41c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:36.122927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 598/1163] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:32:36.400366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196ebf2-e69d-433c-9b5d-e31bb5060ea5 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:36.510700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196ebf2-e69d-433c-9b5d-e31bb5060ea5 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:36.548030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3580b95-0d3a-4b08-9a9a-4e153bccfa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:36.548642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3580b95-0d3a-4b08-9a9a-4e153bccfa88 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:36.548916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3580b95-0d3a-4b08-9a9a-4e153bccfa88 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:36.549180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3580b95-0d3a-4b08-9a9a-4e153bccfa88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:36.614242 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:36.622078 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3580b95-0d3a-4b08-9a9a-4e153bccfa88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:36.623578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3580b95-0d3a-4b08-9a9a-4e153bccfa88 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:36.737841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 566/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:32:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 23:32:36.744839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b639d-2da5-4a88-b5c1-734af8ad41c6 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:36.857024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b639d-2da5-4a88-b5c1-734af8ad41c6 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:36.886903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:36.894916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100521c5-9f9e-4e16-a63b-1c8e508e23d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:36.896457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100521c5-9f9e-4e16-a63b-1c8e508e23d7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:36.896457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100521c5-9f9e-4e16-a63b-1c8e508e23d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:36.896457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100521c5-9f9e-4e16-a63b-1c8e508e23d7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:36.903376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100521c5-9f9e-4e16-a63b-1c8e508e23d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:36.904886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100521c5-9f9e-4e16-a63b-1c8e508e23d7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-100521c5-9f9e-4e16-a63b-1c8e508e23d7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:36.916736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:36.918240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:32:36.919634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 599/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Aug 29 23:32:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:32:36.925071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3580b95-0d3a-4b08-9a9a-4e153bccfa88 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:37.032471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:37.039282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3580b95-0d3a-4b08-9a9a-4e153bccfa88 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:37.081494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579dd75-7f61-4998-ac71-56ef1c7e3694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:37.082117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579dd75-7f61-4998-ac71-56ef1c7e3694 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:37.082398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579dd75-7f61-4998-ac71-56ef1c7e3694 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:37.082707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579dd75-7f61-4998-ac71-56ef1c7e3694 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:37.083381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579dd75-7f61-4998-ac71-56ef1c7e3694 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:37.083688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579dd75-7f61-4998-ac71-56ef1c7e3694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:37.085167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579dd75-7f61-4998-ac71-56ef1c7e3694 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:37.093780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 567/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Aug 29 23:32:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:39.088218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817ab2-64ce-4bce-93f5-405cb189bb94 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:39.088796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817ab2-64ce-4bce-93f5-405cb189bb94 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:39.088796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817ab2-64ce-4bce-93f5-405cb189bb94 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:39.090155 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 600/1167] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:32:39.095896 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90110ba-f8de-4ad2-8725-7bfe73aa9b49 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:39.096373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90110ba-f8de-4ad2-8725-7bfe73aa9b49 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:39.096481 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90110ba-f8de-4ad2-8725-7bfe73aa9b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:39.115210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-c90110ba-f8de-4ad2-8725-7bfe73aa9b49 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:39.117423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c90110ba-f8de-4ad2-8725-7bfe73aa9b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:39.188349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 568/1168] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:32:39.462732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc817ab2-64ce-4bce-93f5-405cb189bb94 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:39.576101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc817ab2-64ce-4bce-93f5-405cb189bb94 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:39.611831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510ff52-6fda-4418-9189-7dbae25b2611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:39.612418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510ff52-6fda-4418-9189-7dbae25b2611 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:39.612738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510ff52-6fda-4418-9189-7dbae25b2611 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:39.613052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510ff52-6fda-4418-9189-7dbae25b2611 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:39.613762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510ff52-6fda-4418-9189-7dbae25b2611 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:39.615204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510ff52-6fda-4418-9189-7dbae25b2611 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:39.696055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 601/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Mon Aug 29 23:32:39 2022] POST /identity/v3/domains => generated 267 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:32:41.763771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebeb2f5-22d7-4c28-98e9-8fa8483d983b None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:41.764306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebeb2f5-22d7-4c28-98e9-8fa8483d983b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:41.764655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebeb2f5-22d7-4c28-98e9-8fa8483d983b None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:41.767205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 569/1170] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:32:41.773177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59363b-5f13-4423-9967-8be30912982e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:41.773743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59363b-5f13-4423-9967-8be30912982e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:41.773743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59363b-5f13-4423-9967-8be30912982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:41.792690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-af59363b-5f13-4423-9967-8be30912982e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:41.793905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-af59363b-5f13-4423-9967-8be30912982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:41.843825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:41.867387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 602/1171] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:32:42.146182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebeb2f5-22d7-4c28-98e9-8fa8483d983b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:42.242207 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:42.253090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebeb2f5-22d7-4c28-98e9-8fa8483d983b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:42.295028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f696a-3400-4b08-9b1c-c9f3b8ea3ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:42.295680 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f696a-3400-4b08-9b1c-c9f3b8ea3ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:42.295958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f696a-3400-4b08-9b1c-c9f3b8ea3ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:42.296230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f696a-3400-4b08-9b1c-c9f3b8ea3ba0 None admin] PATH_INFO: `/v3/domains/89f60f2621994b14be286165d6fb4756` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:42.309511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f696a-3400-4b08-9b1c-c9f3b8ea3ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=89f60f2621994b14be286165d6fb4756)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:42.311036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f696a-3400-4b08-9b1c-c9f3b8ea3ba0 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:42.312808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 570/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Mon Aug 29 23:32:42 2022] GET /identity/v3/domains/89f60f2621994b14be286165d6fb4756 => generated 267 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:42.320202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59363b-5f13-4423-9967-8be30912982e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:42.433636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59363b-5f13-4423-9967-8be30912982e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:42.469917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea32fb2-9b2e-4e3b-b796-199ad502e057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:42.470560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea32fb2-9b2e-4e3b-b796-199ad502e057 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:42.470841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea32fb2-9b2e-4e3b-b796-199ad502e057 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:42.471114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea32fb2-9b2e-4e3b-b796-199ad502e057 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:42.471780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea32fb2-9b2e-4e3b-b796-199ad502e057 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:42.473226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea32fb2-9b2e-4e3b-b796-199ad502e057 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:42.655899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 603/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:32:42 2022] POST /identity/v3/projects => generated 361 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:32:44.703415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22140d01-d8b7-4da4-972e-7177a23b39ba None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:44.703965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22140d01-d8b7-4da4-972e-7177a23b39ba None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:44.703965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22140d01-d8b7-4da4-972e-7177a23b39ba None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:44.705353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 571/1174] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:32:44.712216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95db3c1-2dd4-4d06-9ac2-69d951745fd1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:44.712784 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95db3c1-2dd4-4d06-9ac2-69d951745fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:44.712784 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95db3c1-2dd4-4d06-9ac2-69d951745fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:44.733062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e95db3c1-2dd4-4d06-9ac2-69d951745fd1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:44.734354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e95db3c1-2dd4-4d06-9ac2-69d951745fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:44.761160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:44.811383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 604/1175] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:32:45.086957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22140d01-d8b7-4da4-972e-7177a23b39ba None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:45.200275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22140d01-d8b7-4da4-972e-7177a23b39ba None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:45.249088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90578cd-a6f4-429c-9013-d83b797452da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:45.250048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90578cd-a6f4-429c-9013-d83b797452da None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:45.250465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90578cd-a6f4-429c-9013-d83b797452da None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:45.250862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90578cd-a6f4-429c-9013-d83b797452da None admin] PATH_INFO: `/v3/domains/89f60f2621994b14be286165d6fb4756` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:45.252573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90578cd-a6f4-429c-9013-d83b797452da None admin] RBAC: Authorizing `identity:get_domain(domain_id=89f60f2621994b14be286165d6fb4756)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:45.254718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90578cd-a6f4-429c-9013-d83b797452da None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:45.257946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 572/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Mon Aug 29 23:32:45 2022] GET /identity/v3/domains/89f60f2621994b14be286165d6fb4756 => generated 267 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:45.264277 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95db3c1-2dd4-4d06-9ac2-69d951745fd1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:45.373430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95db3c1-2dd4-4d06-9ac2-69d951745fd1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:45.409546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c99f86-bfbc-4242-8d90-4be0af7f0e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:45.410157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c99f86-bfbc-4242-8d90-4be0af7f0e80 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:45.410460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c99f86-bfbc-4242-8d90-4be0af7f0e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:45.410735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c99f86-bfbc-4242-8d90-4be0af7f0e80 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:45.411403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c99f86-bfbc-4242-8d90-4be0af7f0e80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:45.413247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c99f86-bfbc-4242-8d90-4be0af7f0e80 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:45.550839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 605/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:32:45 2022] POST /identity/v3/users => generated 311 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:32:47.597242 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d885e17-f4e0-4df6-8994-410deddf4197 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:47.597849 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d885e17-f4e0-4df6-8994-410deddf4197 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:47.597849 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d885e17-f4e0-4df6-8994-410deddf4197 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:47.599229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 573/1178] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:32:47.606288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb877bf-7fcf-4687-a9a1-7658eb9ad220 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:47.606925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb877bf-7fcf-4687-a9a1-7658eb9ad220 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:47.606925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb877bf-7fcf-4687-a9a1-7658eb9ad220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:47.627183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3bb877bf-7fcf-4687-a9a1-7658eb9ad220 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:47.628423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3bb877bf-7fcf-4687-a9a1-7658eb9ad220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:47.642012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:47.703541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 606/1179] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 23:32:47.987085 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d885e17-f4e0-4df6-8994-410deddf4197 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:48.008614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:48.104341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d885e17-f4e0-4df6-8994-410deddf4197 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:48.143447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a65fe-2f85-4ff9-86aa-0aa8a92ca9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:48.144077 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a65fe-2f85-4ff9-86aa-0aa8a92ca9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:48.144353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a65fe-2f85-4ff9-86aa-0aa8a92ca9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:48.144621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a65fe-2f85-4ff9-86aa-0aa8a92ca9c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:48.157360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a65fe-2f85-4ff9-86aa-0aa8a92ca9c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-492a65fe-2f85-4ff9-86aa-0aa8a92ca9c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:32:48.163380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:48.165107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:32:48.166324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 574/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:32:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:32:48.170968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb877bf-7fcf-4687-a9a1-7658eb9ad220 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:48.282745 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb877bf-7fcf-4687-a9a1-7658eb9ad220 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:48.320196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0ac91-e9fc-4e3d-b177-b0b2638c281e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:48.320968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0ac91-e9fc-4e3d-b177-b0b2638c281e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:48.321305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0ac91-e9fc-4e3d-b177-b0b2638c281e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:48.321635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0ac91-e9fc-4e3d-b177-b0b2638c281e None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:48.322528 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0ac91-e9fc-4e3d-b177-b0b2638c281e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:48.322963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0ac91-e9fc-4e3d-b177-b0b2638c281e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:48.324523 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0ac91-e9fc-4e3d-b177-b0b2638c281e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:48.334400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 607/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:32:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:48.340694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a65fe-2f85-4ff9-86aa-0aa8a92ca9c8 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:48.429582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:48.457333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a65fe-2f85-4ff9-86aa-0aa8a92ca9c8 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:48.496403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b076c-345a-4166-a278-840824ea6f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:48.497059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b076c-345a-4166-a278-840824ea6f52 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:48.497338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b076c-345a-4166-a278-840824ea6f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:48.497620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b076c-345a-4166-a278-840824ea6f52 None admin] PATH_INFO: `/v3/users/377040391b54495b8a38e5d9acd1b325` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:48.514799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b076c-345a-4166-a278-840824ea6f52 None admin] RBAC: Authorizing `identity:get_user(user_id=377040391b54495b8a38e5d9acd1b325)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:48.516356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b076c-345a-4166-a278-840824ea6f52 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:48.518223 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 575/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:48 2022] GET /identity/v3/users/377040391b54495b8a38e5d9acd1b325 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:48.525388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0ac91-e9fc-4e3d-b177-b0b2638c281e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:48.538530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:48.644289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0ac91-e9fc-4e3d-b177-b0b2638c281e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:48.681939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc2c03-52d6-4c31-9784-d5e2352e669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:48.682641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc2c03-52d6-4c31-9784-d5e2352e669f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:48.682968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc2c03-52d6-4c31-9784-d5e2352e669f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:48.683286 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc2c03-52d6-4c31-9784-d5e2352e669f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:48.759426 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc2c03-52d6-4c31-9784-d5e2352e669f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:48.761160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc2c03-52d6-4c31-9784-d5e2352e669f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:48.772585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:48.875622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 608/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:32:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:32:48.883296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b076c-345a-4166-a278-840824ea6f52 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:48.995792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b076c-345a-4166-a278-840824ea6f52 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:49.031887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e45587-78fe-4ca3-b7bf-238e3f785a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:49.032507 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e45587-78fe-4ca3-b7bf-238e3f785a3b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:49.032783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e45587-78fe-4ca3-b7bf-238e3f785a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:49.033054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e45587-78fe-4ca3-b7bf-238e3f785a3b None admin] PATH_INFO: `/v3/projects/5085944e4199429fbcf4319bf3747ffb` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:49.034200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e45587-78fe-4ca3-b7bf-238e3f785a3b None admin] RBAC: Authorizing `identity:get_project(project_id=5085944e4199429fbcf4319bf3747ffb)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:49.035649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e45587-78fe-4ca3-b7bf-238e3f785a3b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:49.038713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 576/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:32:48 2022] GET /identity/v3/projects/5085944e4199429fbcf4319bf3747ffb => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:49.044906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc2c03-52d6-4c31-9784-d5e2352e669f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:49.151925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc2c03-52d6-4c31-9784-d5e2352e669f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:49.162079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:49.187894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04150e5b-3bc9-4376-92fb-6fe70ca8b6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:49.188548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04150e5b-3bc9-4376-92fb-6fe70ca8b6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:49.188867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04150e5b-3bc9-4376-92fb-6fe70ca8b6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:49.189185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04150e5b-3bc9-4376-92fb-6fe70ca8b6f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:49.189968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04150e5b-3bc9-4376-92fb-6fe70ca8b6f0 None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=5085944e4199429fbcf4319bf3747ffb, user.id=377040391b54495b8a38e5d9acd1b325) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:49.190342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04150e5b-3bc9-4376-92fb-6fe70ca8b6f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:49.191895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04150e5b-3bc9-4376-92fb-6fe70ca8b6f0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:49.199014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 609/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:32:49 2022] GET /identity/v3/role_assignments?user.id=377040391b54495b8a38e5d9acd1b325&scope.project.id=5085944e4199429fbcf4319bf3747ffb&role.id=d86dba8efe20423abdde06031690beb8 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:51.256090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e4fc6-a957-40ce-bb4e-198e96b75fc2 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:51.256665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e4fc6-a957-40ce-bb4e-198e96b75fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:51.256829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e4fc6-a957-40ce-bb4e-198e96b75fc2 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:51.258136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 577/1186] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:32:51.266554 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34831c-77e3-4713-8637-d6d66c2769ee None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:51.267112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34831c-77e3-4713-8637-d6d66c2769ee None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:51.267112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34831c-77e3-4713-8637-d6d66c2769ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:51.286391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1b34831c-77e3-4713-8637-d6d66c2769ee None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:51.288145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1b34831c-77e3-4713-8637-d6d66c2769ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:51.359831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 610/1187] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:32:51.646918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e4fc6-a957-40ce-bb4e-198e96b75fc2 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:51.651039 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:51.785475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e4fc6-a957-40ce-bb4e-198e96b75fc2 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:51.824497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e02f6-0609-4c08-9cd3-2cbe17ca42aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:51.825123 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e02f6-0609-4c08-9cd3-2cbe17ca42aa None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:51.825397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e02f6-0609-4c08-9cd3-2cbe17ca42aa None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:51.825657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e02f6-0609-4c08-9cd3-2cbe17ca42aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:51.837431 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e02f6-0609-4c08-9cd3-2cbe17ca42aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-719e02f6-0609-4c08-9cd3-2cbe17ca42aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:32:51.842320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:51.843585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:32:51.844539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 578/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:32:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:32:51.850626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34831c-77e3-4713-8637-d6d66c2769ee None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:51.972622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34831c-77e3-4713-8637-d6d66c2769ee None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:51.975347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:52.015150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e6d3f-1f18-4905-9544-d428741b1e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:52.015767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e6d3f-1f18-4905-9544-d428741b1e2f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:52.016041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e6d3f-1f18-4905-9544-d428741b1e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:52.016311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e6d3f-1f18-4905-9544-d428741b1e2f None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:52.016981 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e6d3f-1f18-4905-9544-d428741b1e2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:52.017282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e6d3f-1f18-4905-9544-d428741b1e2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:52.018700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e6d3f-1f18-4905-9544-d428741b1e2f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:52.028246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 611/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:32:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:52.034397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e02f6-0609-4c08-9cd3-2cbe17ca42aa None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:52.106858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:52.150356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e02f6-0609-4c08-9cd3-2cbe17ca42aa None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:52.188039 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ed301-8700-474f-a60b-b84fec54155a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:52.188674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ed301-8700-474f-a60b-b84fec54155a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:52.188954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ed301-8700-474f-a60b-b84fec54155a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:52.189227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ed301-8700-474f-a60b-b84fec54155a None admin] PATH_INFO: `/v3/users/377040391b54495b8a38e5d9acd1b325` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:52.190415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ed301-8700-474f-a60b-b84fec54155a None admin] RBAC: Authorizing `identity:get_user(user_id=377040391b54495b8a38e5d9acd1b325)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:52.191882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ed301-8700-474f-a60b-b84fec54155a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:52.193576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 579/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:52 2022] GET /identity/v3/users/377040391b54495b8a38e5d9acd1b325 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:52.199760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e6d3f-1f18-4905-9544-d428741b1e2f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:52.329894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e6d3f-1f18-4905-9544-d428741b1e2f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:52.369880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c90c2-a27a-4ec0-9d51-5ceea6452479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:52.370648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c90c2-a27a-4ec0-9d51-5ceea6452479 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:52.370982 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c90c2-a27a-4ec0-9d51-5ceea6452479 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:52.371320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c90c2-a27a-4ec0-9d51-5ceea6452479 None admin] PATH_INFO: `/v3/projects/5085944e4199429fbcf4319bf3747ffb` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:52.372496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c90c2-a27a-4ec0-9d51-5ceea6452479 None admin] RBAC: Authorizing `identity:get_project(project_id=5085944e4199429fbcf4319bf3747ffb)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:52.374051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c90c2-a27a-4ec0-9d51-5ceea6452479 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:52.375821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 612/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:32:52 2022] GET /identity/v3/projects/5085944e4199429fbcf4319bf3747ffb => generated 361 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:52.382882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ed301-8700-474f-a60b-b84fec54155a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:52.492678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ed301-8700-474f-a60b-b84fec54155a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:52.525154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:52.530367 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba2900-e999-409d-bccc-0d15519c86f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:52.531068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba2900-e999-409d-bccc-0d15519c86f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:52.531345 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba2900-e999-409d-bccc-0d15519c86f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:52.531615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba2900-e999-409d-bccc-0d15519c86f5 None admin] PATH_INFO: `/v3/projects/5085944e4199429fbcf4319bf3747ffb/users/377040391b54495b8a38e5d9acd1b325/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:52.533596 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba2900-e999-409d-bccc-0d15519c86f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5085944e4199429fbcf4319bf3747ffb, user_id=377040391b54495b8a38e5d9acd1b325, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:52.535075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba2900-e999-409d-bccc-0d15519c86f5 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:52.636786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 580/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:32:52 2022] PUT /identity/v3/projects/5085944e4199429fbcf4319bf3747ffb/users/377040391b54495b8a38e5d9acd1b325/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:32:54.614137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268d769-2e38-428a-b262-9ef0aff8367f None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:54.615344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268d769-2e38-428a-b262-9ef0aff8367f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:54.615344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268d769-2e38-428a-b262-9ef0aff8367f None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:54.616816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 613/1193] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:32:54.623417 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d749316-740c-457d-9de3-e7a4f0360507 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:54.623922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d749316-740c-457d-9de3-e7a4f0360507 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:54.624025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d749316-740c-457d-9de3-e7a4f0360507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:54.643682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7d749316-740c-457d-9de3-e7a4f0360507 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:32:54.644942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7d749316-740c-457d-9de3-e7a4f0360507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:32:54.718389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 581/1194] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:32:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:32:54.991923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268d769-2e38-428a-b262-9ef0aff8367f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:55.026951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:55.102971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268d769-2e38-428a-b262-9ef0aff8367f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:55.140416 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe0bb0-9495-441b-877a-1609f4e3dad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:55.141095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe0bb0-9495-441b-877a-1609f4e3dad7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:55.141422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe0bb0-9495-441b-877a-1609f4e3dad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:55.141741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe0bb0-9495-441b-877a-1609f4e3dad7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:55.153596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe0bb0-9495-441b-877a-1609f4e3dad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-dffe0bb0-9495-441b-877a-1609f4e3dad7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:32:55.158360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:32:55.160124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:32:55.161414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 614/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:32:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:32:55.165976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d749316-740c-457d-9de3-e7a4f0360507 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:55.269692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d749316-740c-457d-9de3-e7a4f0360507 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:55.294309 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:55.307611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6453b-57e4-4950-ad47-f319476fae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:55.308416 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6453b-57e4-4950-ad47-f319476fae8c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:55.308785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6453b-57e4-4950-ad47-f319476fae8c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:55.309163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6453b-57e4-4950-ad47-f319476fae8c None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:55.309939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6453b-57e4-4950-ad47-f319476fae8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:55.310384 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6453b-57e4-4950-ad47-f319476fae8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:55.312646 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6453b-57e4-4950-ad47-f319476fae8c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:55.324819 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 582/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:32:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:32:55.332143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe0bb0-9495-441b-877a-1609f4e3dad7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:55.437886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:55.447899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe0bb0-9495-441b-877a-1609f4e3dad7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:55.484174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5d4be-6734-4cd1-850b-db747dbb257e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:55.484799 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5d4be-6734-4cd1-850b-db747dbb257e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:55.485081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5d4be-6734-4cd1-850b-db747dbb257e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:55.485349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5d4be-6734-4cd1-850b-db747dbb257e None admin] PATH_INFO: `/v3/users/377040391b54495b8a38e5d9acd1b325` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:55.486556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5d4be-6734-4cd1-850b-db747dbb257e None admin] RBAC: Authorizing `identity:get_user(user_id=377040391b54495b8a38e5d9acd1b325)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:55.487996 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5d4be-6734-4cd1-850b-db747dbb257e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:55.489637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 615/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Aug 29 23:32:55 2022] GET /identity/v3/users/377040391b54495b8a38e5d9acd1b325 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:55.495789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6453b-57e4-4950-ad47-f319476fae8c None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:55.603846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6453b-57e4-4950-ad47-f319476fae8c None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:55.648006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603748ec-6635-4ccf-97ff-fb26f9c62bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:55.648694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603748ec-6635-4ccf-97ff-fb26f9c62bea None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:55.648889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603748ec-6635-4ccf-97ff-fb26f9c62bea None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:55.649161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603748ec-6635-4ccf-97ff-fb26f9c62bea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:55.707806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:55.724845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603748ec-6635-4ccf-97ff-fb26f9c62bea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:55.726548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603748ec-6635-4ccf-97ff-fb26f9c62bea None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:55.838312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 583/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:32:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:32:55.844548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5d4be-6734-4cd1-850b-db747dbb257e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:55.955665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5d4be-6734-4cd1-850b-db747dbb257e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:55.999448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de1669-65b9-441a-b42b-ea0fb5f03f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:56.000088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de1669-65b9-441a-b42b-ea0fb5f03f09 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:56.000361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de1669-65b9-441a-b42b-ea0fb5f03f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:56.000629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de1669-65b9-441a-b42b-ea0fb5f03f09 None admin] PATH_INFO: `/v3/projects/5085944e4199429fbcf4319bf3747ffb` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:56.001781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de1669-65b9-441a-b42b-ea0fb5f03f09 None admin] RBAC: Authorizing `identity:get_project(project_id=5085944e4199429fbcf4319bf3747ffb)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:56.003267 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de1669-65b9-441a-b42b-ea0fb5f03f09 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:56.005441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 616/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Aug 29 23:32:55 2022] GET /identity/v3/projects/5085944e4199429fbcf4319bf3747ffb => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:32:56.015908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603748ec-6635-4ccf-97ff-fb26f9c62bea None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:32:56.122980 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:32:56.140091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603748ec-6635-4ccf-97ff-fb26f9c62bea None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:32:56.176258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d41121-6cfa-4b02-9cff-208ac1ed076c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:32:56.176973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d41121-6cfa-4b02-9cff-208ac1ed076c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:32:56.177316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d41121-6cfa-4b02-9cff-208ac1ed076c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:32:56.177614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d41121-6cfa-4b02-9cff-208ac1ed076c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:32:56.178405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d41121-6cfa-4b02-9cff-208ac1ed076c None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=5085944e4199429fbcf4319bf3747ffb, user.id=377040391b54495b8a38e5d9acd1b325) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:32:56.178786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d41121-6cfa-4b02-9cff-208ac1ed076c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:32:56.180281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d41121-6cfa-4b02-9cff-208ac1ed076c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:32:56.186654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 584/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:32:56 2022] GET /identity/v3/role_assignments?user.id=377040391b54495b8a38e5d9acd1b325&scope.project.id=5085944e4199429fbcf4319bf3747ffb&role.id=d86dba8efe20423abdde06031690beb8 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:33:05.994816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=86 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:33:06.183767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=26 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:11.214521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b3086-bb69-431f-a737-eef685652ac0 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:11.215113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b3086-bb69-431f-a737-eef685652ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:11.215113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b3086-bb69-431f-a737-eef685652ac0 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:11.216399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 617/1201] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:34:11.222700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c2c5c-98aa-4094-af98-87efa646f806 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:11.223144 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c2c5c-98aa-4094-af98-87efa646f806 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:11.223271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c2c5c-98aa-4094-af98-87efa646f806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:11.242675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-c88c2c5c-98aa-4094-af98-87efa646f806 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:34:11.243869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c88c2c5c-98aa-4094-af98-87efa646f806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:34:11.316842 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 585/1202] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:34:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:34:11.593602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b3086-bb69-431f-a737-eef685652ac0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:11.700903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b3086-bb69-431f-a737-eef685652ac0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:11.736544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c6054-3767-4875-bd96-d9d25009ea8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:11.737171 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6054-3767-4875-bd96-d9d25009ea8a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:11.737542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6054-3767-4875-bd96-d9d25009ea8a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:11.737820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6054-3767-4875-bd96-d9d25009ea8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:11.749382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c6054-3767-4875-bd96-d9d25009ea8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:11.750872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c6054-3767-4875-bd96-d9d25009ea8a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-366c6054-3767-4875-bd96-d9d25009ea8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:11.765310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:11.766870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:11.768001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:11.768001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:11.768001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:11.768001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 618/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:11.775011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c2c5c-98aa-4094-af98-87efa646f806 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:11.876208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c2c5c-98aa-4094-af98-87efa646f806 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:11.906613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:11.913030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921d999-46bb-43b6-84df-a1835a616334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:11.913784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921d999-46bb-43b6-84df-a1835a616334 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:11.914154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921d999-46bb-43b6-84df-a1835a616334 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:11.914592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921d999-46bb-43b6-84df-a1835a616334 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:11.917920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921d999-46bb-43b6-84df-a1835a616334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:11.917920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921d999-46bb-43b6-84df-a1835a616334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:11.917920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921d999-46bb-43b6-84df-a1835a616334 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:11.928508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 586/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:11.934464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c6054-3767-4875-bd96-d9d25009ea8a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:11.989827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:12.036305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c6054-3767-4875-bd96-d9d25009ea8a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:12.071649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15a220-4d2a-436a-8996-c44c60f40d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:12.072261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15a220-4d2a-436a-8996-c44c60f40d37 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:12.072541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15a220-4d2a-436a-8996-c44c60f40d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:12.072814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15a220-4d2a-436a-8996-c44c60f40d37 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:12.073467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15a220-4d2a-436a-8996-c44c60f40d37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:12.074993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15a220-4d2a-436a-8996-c44c60f40d37 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:12.191683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 619/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Aug 29 23:34:11 2022] POST /identity/v3/users => generated 251 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:34:14.160386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af076c-9382-4b0b-b32e-bb970cb07525 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:14.161012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af076c-9382-4b0b-b32e-bb970cb07525 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:14.161012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af076c-9382-4b0b-b32e-bb970cb07525 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:14.162313 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 587/1206] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:34:14.168391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444976d-f9c4-48a1-8f81-f6d3ed5bf762 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:14.168936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444976d-f9c4-48a1-8f81-f6d3ed5bf762 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:14.169034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444976d-f9c4-48a1-8f81-f6d3ed5bf762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:14.189052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f444976d-f9c4-48a1-8f81-f6d3ed5bf762 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:34:14.194167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f444976d-f9c4-48a1-8f81-f6d3ed5bf762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:34:14.266751 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 620/1207] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:34:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:34:14.541126 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af076c-9382-4b0b-b32e-bb970cb07525 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:14.647882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af076c-9382-4b0b-b32e-bb970cb07525 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:14.682466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdc065-70e8-42a5-b5fd-5ea787c35003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:14.683089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdc065-70e8-42a5-b5fd-5ea787c35003 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:14.683362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdc065-70e8-42a5-b5fd-5ea787c35003 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:14.683629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdc065-70e8-42a5-b5fd-5ea787c35003 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:14.695308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdc065-70e8-42a5-b5fd-5ea787c35003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-f3cdc065-70e8-42a5-b5fd-5ea787c35003 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:34:14.700120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:14.701353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:14.702538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 588/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:34:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:14.707769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444976d-f9c4-48a1-8f81-f6d3ed5bf762 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:14.748923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:14.808383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444976d-f9c4-48a1-8f81-f6d3ed5bf762 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:14.845972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92767da-d743-4ebb-b9bb-5b33afc4f54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:14.846644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92767da-d743-4ebb-b9bb-5b33afc4f54f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:14.846923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92767da-d743-4ebb-b9bb-5b33afc4f54f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:14.847229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92767da-d743-4ebb-b9bb-5b33afc4f54f None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:14.848107 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92767da-d743-4ebb-b9bb-5b33afc4f54f None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:14.848379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92767da-d743-4ebb-b9bb-5b33afc4f54f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:14.849774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92767da-d743-4ebb-b9bb-5b33afc4f54f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:14.858140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 621/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:34:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:14.864565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdc065-70e8-42a5-b5fd-5ea787c35003 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:14.916404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:14.968965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdc065-70e8-42a5-b5fd-5ea787c35003 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:15.002955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25e1c1-d821-41b3-9f27-4e9237b58f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:15.003585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25e1c1-d821-41b3-9f27-4e9237b58f70 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:15.003864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25e1c1-d821-41b3-9f27-4e9237b58f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:15.004136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25e1c1-d821-41b3-9f27-4e9237b58f70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:15.015779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25e1c1-d821-41b3-9f27-4e9237b58f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:15.017222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25e1c1-d821-41b3-9f27-4e9237b58f70 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-7f25e1c1-d821-41b3-9f27-4e9237b58f70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:15.031413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:15.032775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:15.033916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:15.033916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:15.033916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:15.033916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 589/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:15.038662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92767da-d743-4ebb-b9bb-5b33afc4f54f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:15.143355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:15.146329 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92767da-d743-4ebb-b9bb-5b33afc4f54f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:15.181512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcebef6-6a1e-4c3f-8944-aabe4818d84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:15.182187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcebef6-6a1e-4c3f-8944-aabe4818d84a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:15.182384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcebef6-6a1e-4c3f-8944-aabe4818d84a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:15.182686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcebef6-6a1e-4c3f-8944-aabe4818d84a None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:15.183326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcebef6-6a1e-4c3f-8944-aabe4818d84a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:15.183630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcebef6-6a1e-4c3f-8944-aabe4818d84a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:15.185008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcebef6-6a1e-4c3f-8944-aabe4818d84a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:15.197364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 622/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:15.203389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25e1c1-d821-41b3-9f27-4e9237b58f70 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:15.308349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25e1c1-d821-41b3-9f27-4e9237b58f70 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:15.309887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:15.344043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada2240-a863-4537-b344-45f37a3792e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:15.344762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada2240-a863-4537-b344-45f37a3792e7 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:15.345040 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada2240-a863-4537-b344-45f37a3792e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:15.345314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada2240-a863-4537-b344-45f37a3792e7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:15.353213 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada2240-a863-4537-b344-45f37a3792e7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:15.354687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada2240-a863-4537-b344-45f37a3792e7 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-cada2240-a863-4537-b344-45f37a3792e7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:34:15.366293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:15.367447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:15.368544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:15.368544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:15.368544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:34:15.368544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:34:15.368544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:34:15.368544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:34:15.368544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:34:15.368544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:34:15.368544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:15.368544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:15.368544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 590/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:15.373496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcebef6-6a1e-4c3f-8944-aabe4818d84a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:15.483925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcebef6-6a1e-4c3f-8944-aabe4818d84a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:15.518615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d78ab-bd4b-4fff-8246-1567b2f17b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:15.519222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d78ab-bd4b-4fff-8246-1567b2f17b3a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:15.519499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d78ab-bd4b-4fff-8246-1567b2f17b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:15.519770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d78ab-bd4b-4fff-8246-1567b2f17b3a None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:15.520519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d78ab-bd4b-4fff-8246-1567b2f17b3a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:15.520824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d78ab-bd4b-4fff-8246-1567b2f17b3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:15.522267 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d78ab-bd4b-4fff-8246-1567b2f17b3a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:15.539929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 623/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Aug 29 23:34:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:15.545825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada2240-a863-4537-b344-45f37a3792e7 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:15.647245 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada2240-a863-4537-b344-45f37a3792e7 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:15.682488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d7247-368d-4c7f-80c7-261354e53c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:15.683508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d7247-368d-4c7f-80c7-261354e53c48 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:15.683983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d7247-368d-4c7f-80c7-261354e53c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:15.684527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d7247-368d-4c7f-80c7-261354e53c48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:15.718594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:15.759828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d7247-368d-4c7f-80c7-261354e53c48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:15.761242 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d7247-368d-4c7f-80c7-261354e53c48 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:15.862724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 591/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:34:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:34:15.868804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d78ab-bd4b-4fff-8246-1567b2f17b3a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:15.889110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:15.969367 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d78ab-bd4b-4fff-8246-1567b2f17b3a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:16.004045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc591ad7-4bc8-4584-94de-d66ff0d91804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:16.004671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc591ad7-4bc8-4584-94de-d66ff0d91804 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:16.004949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc591ad7-4bc8-4584-94de-d66ff0d91804 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:16.005222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc591ad7-4bc8-4584-94de-d66ff0d91804 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:16.015588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc591ad7-4bc8-4584-94de-d66ff0d91804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:16.016979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc591ad7-4bc8-4584-94de-d66ff0d91804 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-cc591ad7-4bc8-4584-94de-d66ff0d91804 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:16.031698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:16.032935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:16.033941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:16.033941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:16.033941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:16.033941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 624/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:16.038970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d7247-368d-4c7f-80c7-261354e53c48 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:16.101866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:16.140439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d7247-368d-4c7f-80c7-261354e53c48 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:16.176415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1b95b-272e-4aeb-9a96-2a2492a758f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:16.181114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1b95b-272e-4aeb-9a96-2a2492a758f2 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:16.181455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1b95b-272e-4aeb-9a96-2a2492a758f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:16.181736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1b95b-272e-4aeb-9a96-2a2492a758f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:16.182395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1b95b-272e-4aeb-9a96-2a2492a758f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:16.182725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1b95b-272e-4aeb-9a96-2a2492a758f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:16.184085 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1b95b-272e-4aeb-9a96-2a2492a758f2 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:16.196410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 592/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:16.202570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc591ad7-4bc8-4584-94de-d66ff0d91804 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:16.285363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:16.316590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc591ad7-4bc8-4584-94de-d66ff0d91804 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:16.350718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cf0b8-3592-46cd-b403-f6e551b5f5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:16.351351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cf0b8-3592-46cd-b403-f6e551b5f5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:16.351651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cf0b8-3592-46cd-b403-f6e551b5f5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:16.351920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cf0b8-3592-46cd-b403-f6e551b5f5f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:16.360537 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6cf0b8-3592-46cd-b403-f6e551b5f5f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:16.363313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6cf0b8-3592-46cd-b403-f6e551b5f5f8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-be6cf0b8-3592-46cd-b403-f6e551b5f5f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:34:16.376415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:16.383238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:16.384312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:34:16.384312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:34:16.384312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:34:16.384312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:34:16.384312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:34:16.384312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:34:16.384312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:16.384312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:16.384312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 625/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:34:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:16.386110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1b95b-272e-4aeb-9a96-2a2492a758f2 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:16.504447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1b95b-272e-4aeb-9a96-2a2492a758f2 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:16.528253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:16.540730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3b612-7442-4843-a7fa-54aec333ab3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:16.541404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3b612-7442-4843-a7fa-54aec333ab3d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:16.541744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3b612-7442-4843-a7fa-54aec333ab3d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:16.542015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3b612-7442-4843-a7fa-54aec333ab3d None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:16.542745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3b612-7442-4843-a7fa-54aec333ab3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:16.543044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3b612-7442-4843-a7fa-54aec333ab3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:16.544488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3b612-7442-4843-a7fa-54aec333ab3d None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:16.561287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 593/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:34:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:16.567891 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cf0b8-3592-46cd-b403-f6e551b5f5f8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:16.682447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cf0b8-3592-46cd-b403-f6e551b5f5f8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:16.712309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:16.719455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6db473-a0b3-4246-b728-b8d15b8d7536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:16.720070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6db473-a0b3-4246-b728-b8d15b8d7536 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:16.720349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6db473-a0b3-4246-b728-b8d15b8d7536 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:16.720620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6db473-a0b3-4246-b728-b8d15b8d7536 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:16.721457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6db473-a0b3-4246-b728-b8d15b8d7536 None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=fb770d4379e74bf79ad4280329c771c6) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:16.722003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6db473-a0b3-4246-b728-b8d15b8d7536 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:16.723503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6db473-a0b3-4246-b728-b8d15b8d7536 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:16.730898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 626/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:34:16 2022] GET /identity/v3/role_assignments?user.id=fb770d4379e74bf79ad4280329c771c6&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:18.708537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5cd62-ec97-4909-bb57-8c18439dad73 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:18.709042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5cd62-ec97-4909-bb57-8c18439dad73 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:18.709042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5cd62-ec97-4909-bb57-8c18439dad73 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:18.710304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 594/1220] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:34:18.715718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da792-e7ba-4daa-8116-5632bd75fd1d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:18.716174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da792-e7ba-4daa-8116-5632bd75fd1d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:18.716276 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da792-e7ba-4daa-8116-5632bd75fd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:18.734040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3a1da792-e7ba-4daa-8116-5632bd75fd1d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:34:18.735298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3a1da792-e7ba-4daa-8116-5632bd75fd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:34:18.812230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 627/1221] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:34:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:34:19.093322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5cd62-ec97-4909-bb57-8c18439dad73 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:19.204136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5cd62-ec97-4909-bb57-8c18439dad73 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:19.242237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fc3b6-fcf2-49ce-99de-f90a14d9216d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:19.242909 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fc3b6-fcf2-49ce-99de-f90a14d9216d None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:19.243189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fc3b6-fcf2-49ce-99de-f90a14d9216d None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:19.243457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fc3b6-fcf2-49ce-99de-f90a14d9216d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:19.255816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fc3b6-fcf2-49ce-99de-f90a14d9216d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-d17fc3b6-fcf2-49ce-99de-f90a14d9216d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:34:19.260995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:19.262125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:19.263487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 595/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:34:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:19.268261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1da792-e7ba-4daa-8116-5632bd75fd1d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:19.371999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1da792-e7ba-4daa-8116-5632bd75fd1d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:19.396084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:19.407205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726934d3-47c8-42bb-9ec1-9572321c41a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:19.407818 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726934d3-47c8-42bb-9ec1-9572321c41a3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:19.408106 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726934d3-47c8-42bb-9ec1-9572321c41a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:19.408381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726934d3-47c8-42bb-9ec1-9572321c41a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:19.409045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726934d3-47c8-42bb-9ec1-9572321c41a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:19.409348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726934d3-47c8-42bb-9ec1-9572321c41a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:19.410773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726934d3-47c8-42bb-9ec1-9572321c41a3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:19.419085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 628/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:34:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:19.425850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fc3b6-fcf2-49ce-99de-f90a14d9216d None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:19.471846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:19.530500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fc3b6-fcf2-49ce-99de-f90a14d9216d None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:19.565334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b622cf-a198-403b-b80c-13b17860486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:19.565951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b622cf-a198-403b-b80c-13b17860486f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:19.566308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b622cf-a198-403b-b80c-13b17860486f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:19.566612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b622cf-a198-403b-b80c-13b17860486f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:19.577201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b622cf-a198-403b-b80c-13b17860486f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:19.578662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b622cf-a198-403b-b80c-13b17860486f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-a4b622cf-a198-403b-b80c-13b17860486f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:19.593649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:19.594800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:19.596095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:19.596095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:19.596095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:19.596095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 596/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:19.601712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726934d3-47c8-42bb-9ec1-9572321c41a3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:19.704608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726934d3-47c8-42bb-9ec1-9572321c41a3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:19.738753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9522a-7283-4466-873d-7348dbbaa9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:19.739384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9522a-7283-4466-873d-7348dbbaa9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:19.739622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9522a-7283-4466-873d-7348dbbaa9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:19.739895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9522a-7283-4466-873d-7348dbbaa9f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:19.740536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9522a-7283-4466-873d-7348dbbaa9f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:19.740839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9522a-7283-4466-873d-7348dbbaa9f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:19.742208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9522a-7283-4466-873d-7348dbbaa9f7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:19.756878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 629/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:19.762487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b622cf-a198-403b-b80c-13b17860486f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:19.862486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:19.869271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b622cf-a198-403b-b80c-13b17860486f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:19.902764 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb423d0-c6a5-43ad-b76c-534cab6e87e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:19.903385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb423d0-c6a5-43ad-b76c-534cab6e87e3 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:19.903664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb423d0-c6a5-43ad-b76c-534cab6e87e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:19.903937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb423d0-c6a5-43ad-b76c-534cab6e87e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:19.911148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb423d0-c6a5-43ad-b76c-534cab6e87e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:19.912587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb423d0-c6a5-43ad-b76c-534cab6e87e3 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-8cb423d0-c6a5-43ad-b76c-534cab6e87e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:34:19.923894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:19.925665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:19.930571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:19.930571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:19.930571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:34:19.930571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:34:19.930571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:34:19.930571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:34:19.930571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:34:19.930571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:34:19.930571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:19.930571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:19.930571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 597/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:19.931403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9522a-7283-4466-873d-7348dbbaa9f7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:19.980900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:20.036126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9522a-7283-4466-873d-7348dbbaa9f7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:20.072763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732d1d0-83b2-4acb-880f-5ab5c821d76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:20.073376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732d1d0-83b2-4acb-880f-5ab5c821d76f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:20.073655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732d1d0-83b2-4acb-880f-5ab5c821d76f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:20.073925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732d1d0-83b2-4acb-880f-5ab5c821d76f None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:20.074690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732d1d0-83b2-4acb-880f-5ab5c821d76f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:20.074998 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732d1d0-83b2-4acb-880f-5ab5c821d76f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:20.077243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732d1d0-83b2-4acb-880f-5ab5c821d76f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:20.095038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 630/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Aug 29 23:34:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:20.101025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb423d0-c6a5-43ad-b76c-534cab6e87e3 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:20.211265 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb423d0-c6a5-43ad-b76c-534cab6e87e3 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:20.245850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318e077-5473-466b-9b83-eec69a2b773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:20.246494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e077-5473-466b-9b83-eec69a2b773c None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:20.246886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e077-5473-466b-9b83-eec69a2b773c None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:20.247160 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e077-5473-466b-9b83-eec69a2b773c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:20.260103 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318e077-5473-466b-9b83-eec69a2b773c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:20.261755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318e077-5473-466b-9b83-eec69a2b773c None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-2318e077-5473-466b-9b83-eec69a2b773c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:20.277537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:20.281168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:20.282083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:20.282083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:20.282083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:20.282083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 598/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:20.285561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732d1d0-83b2-4acb-880f-5ab5c821d76f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:20.392078 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732d1d0-83b2-4acb-880f-5ab5c821d76f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:20.394368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:20.425879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b03e2c-3eff-4096-a3d0-00dcf677897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:20.426488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b03e2c-3eff-4096-a3d0-00dcf677897c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:20.426763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b03e2c-3eff-4096-a3d0-00dcf677897c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:20.427029 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b03e2c-3eff-4096-a3d0-00dcf677897c None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:20.427656 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b03e2c-3eff-4096-a3d0-00dcf677897c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:20.427948 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b03e2c-3eff-4096-a3d0-00dcf677897c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:20.429300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b03e2c-3eff-4096-a3d0-00dcf677897c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:20.440319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 631/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:20.446177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318e077-5473-466b-9b83-eec69a2b773c None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:20.456993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:20.549953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318e077-5473-466b-9b83-eec69a2b773c None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:20.585447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db146c18-ef58-42d6-95ad-a6bc5b6ab50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:20.586076 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db146c18-ef58-42d6-95ad-a6bc5b6ab50b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:20.586355 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db146c18-ef58-42d6-95ad-a6bc5b6ab50b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:20.586660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db146c18-ef58-42d6-95ad-a6bc5b6ab50b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:20.594216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db146c18-ef58-42d6-95ad-a6bc5b6ab50b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:20.595750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db146c18-ef58-42d6-95ad-a6bc5b6ab50b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-db146c18-ef58-42d6-95ad-a6bc5b6ab50b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:34:20.606958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:20.607926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:20.609058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:34:20.609058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:34:20.609058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:34:20.609058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:34:20.609058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:34:20.609058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:34:20.609058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:20.609058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:20.609058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 599/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:34:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:20.614102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b03e2c-3eff-4096-a3d0-00dcf677897c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:20.730696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b03e2c-3eff-4096-a3d0-00dcf677897c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:20.768411 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1480792-79ed-4463-960e-4354ce93873a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:20.769138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1480792-79ed-4463-960e-4354ce93873a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:20.769410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1480792-79ed-4463-960e-4354ce93873a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:20.769676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1480792-79ed-4463-960e-4354ce93873a None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:20.770471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1480792-79ed-4463-960e-4354ce93873a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:20.770847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1480792-79ed-4463-960e-4354ce93873a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:20.772275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1480792-79ed-4463-960e-4354ce93873a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:20.785091 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 632/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:34:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:20.791471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db146c18-ef58-42d6-95ad-a6bc5b6ab50b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:20.863714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:20.901582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db146c18-ef58-42d6-95ad-a6bc5b6ab50b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:20.938888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91668bde-24d1-43e1-90f6-f33bb2f6aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:20.939571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668bde-24d1-43e1-90f6-f33bb2f6aabd None admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:20.939853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668bde-24d1-43e1-90f6-f33bb2f6aabd None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:20.940129 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668bde-24d1-43e1-90f6-f33bb2f6aabd None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/fb770d4379e74bf79ad4280329c771c6/roles/f0926be4d3d4462fac854212e101ccdd` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:20.957442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91668bde-24d1-43e1-90f6-f33bb2f6aabd None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=fb770d4379e74bf79ad4280329c771c6, role_id=f0926be4d3d4462fac854212e101ccdd)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:20.958953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91668bde-24d1-43e1-90f6-f33bb2f6aabd None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:21.053522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 600/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:34:20 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/fb770d4379e74bf79ad4280329c771c6/roles/f0926be4d3d4462fac854212e101ccdd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:34:22.975362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d39704-e710-4eee-af48-1af8d75e7529 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:22.976185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d39704-e710-4eee-af48-1af8d75e7529 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:22.976439 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d39704-e710-4eee-af48-1af8d75e7529 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:22.977940 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 633/1233] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:34:22.983435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14903d0a-4dae-4bb2-b46b-2ba0b2c73db6 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:22.983960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14903d0a-4dae-4bb2-b46b-2ba0b2c73db6 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:22.983960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14903d0a-4dae-4bb2-b46b-2ba0b2c73db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:23.002493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-14903d0a-4dae-4bb2-b46b-2ba0b2c73db6 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:34:23.005397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-14903d0a-4dae-4bb2-b46b-2ba0b2c73db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:34:23.070563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 601/1234] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:34:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:34:23.343161 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d39704-e710-4eee-af48-1af8d75e7529 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:23.368863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:23.453058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d39704-e710-4eee-af48-1af8d75e7529 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:23.493601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca7435-15c1-49b3-b20b-104a4cf2d2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:23.494293 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca7435-15c1-49b3-b20b-104a4cf2d2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:23.494590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca7435-15c1-49b3-b20b-104a4cf2d2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:23.494863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca7435-15c1-49b3-b20b-104a4cf2d2c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:23.506048 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca7435-15c1-49b3-b20b-104a4cf2d2c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-8fca7435-15c1-49b3-b20b-104a4cf2d2c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:34:23.510879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:23.512002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:23.513333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 634/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Aug 29 23:34:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:23.518817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14903d0a-4dae-4bb2-b46b-2ba0b2c73db6 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:23.567346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:23.619806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14903d0a-4dae-4bb2-b46b-2ba0b2c73db6 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:23.656225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54590a-67e6-4c91-8e1b-4c406067e42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:23.656833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54590a-67e6-4c91-8e1b-4c406067e42e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:23.657118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54590a-67e6-4c91-8e1b-4c406067e42e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:23.657391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54590a-67e6-4c91-8e1b-4c406067e42e None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:23.658052 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54590a-67e6-4c91-8e1b-4c406067e42e None admin] RBAC: Adding query filter params (name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:23.658357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54590a-67e6-4c91-8e1b-4c406067e42e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:23.659784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54590a-67e6-4c91-8e1b-4c406067e42e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:23.667699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 602/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Aug 29 23:34:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:23.674877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca7435-15c1-49b3-b20b-104a4cf2d2c0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:23.765311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:23.780899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca7435-15c1-49b3-b20b-104a4cf2d2c0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:23.815000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da42a0e-61df-4800-9f10-6a57b506f616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:23.815630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da42a0e-61df-4800-9f10-6a57b506f616 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:23.815903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da42a0e-61df-4800-9f10-6a57b506f616 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:23.816172 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da42a0e-61df-4800-9f10-6a57b506f616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:23.827848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da42a0e-61df-4800-9f10-6a57b506f616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:23.829412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da42a0e-61df-4800-9f10-6a57b506f616 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-6da42a0e-61df-4800-9f10-6a57b506f616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:23.844361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:23.845703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:23.846929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:23.846929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:23.846929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:23.850396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 635/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:23.851412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54590a-67e6-4c91-8e1b-4c406067e42e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:23.953118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54590a-67e6-4c91-8e1b-4c406067e42e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:23.962031 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:23.988791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d447e-ed96-4b6c-b0d9-e8dd148414fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:23.989496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d447e-ed96-4b6c-b0d9-e8dd148414fa None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:23.989670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d447e-ed96-4b6c-b0d9-e8dd148414fa None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:23.989935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d447e-ed96-4b6c-b0d9-e8dd148414fa None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:23.990767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d447e-ed96-4b6c-b0d9-e8dd148414fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:23.991090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d447e-ed96-4b6c-b0d9-e8dd148414fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:23.992498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d447e-ed96-4b6c-b0d9-e8dd148414fa None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:24.004873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 603/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:24.011576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da42a0e-61df-4800-9f10-6a57b506f616 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:24.137173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da42a0e-61df-4800-9f10-6a57b506f616 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:24.174015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b5af8-c94f-4c28-9a27-bcfb78b07ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:24.174715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b5af8-c94f-4c28-9a27-bcfb78b07ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:24.174975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b5af8-c94f-4c28-9a27-bcfb78b07ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:24.175245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b5af8-c94f-4c28-9a27-bcfb78b07ac4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:24.178452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:24.184320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b5af8-c94f-4c28-9a27-bcfb78b07ac4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:24.185830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b5af8-c94f-4c28-9a27-bcfb78b07ac4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e64b5af8-c94f-4c28-9a27-bcfb78b07ac4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:34:24.198022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:24.199251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:24.200547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:24.200547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:24.200547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:34:24.200547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:34:24.200547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:34:24.200547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:34:24.200547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:34:24.200547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:34:24.200547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:24.200547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:24.200547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 636/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:24.209258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d447e-ed96-4b6c-b0d9-e8dd148414fa None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:24.317747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d447e-ed96-4b6c-b0d9-e8dd148414fa None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:24.365755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbf6fc-5a9c-410f-9b55-f1a6de4ea629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:24.366370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbf6fc-5a9c-410f-9b55-f1a6de4ea629 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:24.366693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbf6fc-5a9c-410f-9b55-f1a6de4ea629 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:24.366951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbf6fc-5a9c-410f-9b55-f1a6de4ea629 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:24.367691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbf6fc-5a9c-410f-9b55-f1a6de4ea629 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:24.368000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbf6fc-5a9c-410f-9b55-f1a6de4ea629 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:24.369435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbf6fc-5a9c-410f-9b55-f1a6de4ea629 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:24.394090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 604/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Aug 29 23:34:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:24.399943 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b5af8-c94f-4c28-9a27-bcfb78b07ac4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:24.504607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b5af8-c94f-4c28-9a27-bcfb78b07ac4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:24.538140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b213b-bcae-46dc-ba71-e398ebbc664b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:24.538762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b213b-bcae-46dc-ba71-e398ebbc664b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:24.539137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b213b-bcae-46dc-ba71-e398ebbc664b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:24.539333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b213b-bcae-46dc-ba71-e398ebbc664b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:24.600128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:24.607325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b213b-bcae-46dc-ba71-e398ebbc664b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:24.609635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b213b-bcae-46dc-ba71-e398ebbc664b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:24.715645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 637/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:34:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:34:24.722528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbf6fc-5a9c-410f-9b55-f1a6de4ea629 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:24.750551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:24.833896 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbf6fc-5a9c-410f-9b55-f1a6de4ea629 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:24.874518 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccc1a3-d759-4b89-b2bd-d4291c7b2073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:24.875154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccc1a3-d759-4b89-b2bd-d4291c7b2073 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:24.875437 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccc1a3-d759-4b89-b2bd-d4291c7b2073 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:24.875710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccc1a3-d759-4b89-b2bd-d4291c7b2073 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:24.886809 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccc1a3-d759-4b89-b2bd-d4291c7b2073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:24.888251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccc1a3-d759-4b89-b2bd-d4291c7b2073 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-14ccc1a3-d759-4b89-b2bd-d4291c7b2073 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:24.903586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:24.905381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:24.906590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:24.906590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:24.906590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:24.906590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 605/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:24.911159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b213b-bcae-46dc-ba71-e398ebbc664b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:25.005393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:25.011532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b213b-bcae-46dc-ba71-e398ebbc664b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:25.045415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3982322-da8c-49b5-9308-ee873e3620a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:25.046160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3982322-da8c-49b5-9308-ee873e3620a3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:25.046466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3982322-da8c-49b5-9308-ee873e3620a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:25.046753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3982322-da8c-49b5-9308-ee873e3620a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:25.047403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3982322-da8c-49b5-9308-ee873e3620a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:25.047708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3982322-da8c-49b5-9308-ee873e3620a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:25.049127 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3982322-da8c-49b5-9308-ee873e3620a3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:25.062388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 638/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:25.068325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccc1a3-d759-4b89-b2bd-d4291c7b2073 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:25.151418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:25.173921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccc1a3-d759-4b89-b2bd-d4291c7b2073 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:25.209925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1492e-c5c2-4360-b0d1-3610a778f589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:25.210578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1492e-c5c2-4360-b0d1-3610a778f589 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:25.210855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1492e-c5c2-4360-b0d1-3610a778f589 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:25.211120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1492e-c5c2-4360-b0d1-3610a778f589 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:25.218998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1492e-c5c2-4360-b0d1-3610a778f589 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:25.220455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1492e-c5c2-4360-b0d1-3610a778f589 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-c6e1492e-c5c2-4360-b0d1-3610a778f589 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:34:25.233565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:25.234705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:25.235620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:34:25.235620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:34:25.235620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:34:25.235620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:34:25.235620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:34:25.235620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:34:25.235620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:25.235620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:25.235620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 606/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:34:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:25.240834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3982322-da8c-49b5-9308-ee873e3620a3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:25.343002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3982322-da8c-49b5-9308-ee873e3620a3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:25.378666 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad4036-9608-4b57-b812-97bf6c80c6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:25.380092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad4036-9608-4b57-b812-97bf6c80c6de None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:25.380092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad4036-9608-4b57-b812-97bf6c80c6de None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:25.380092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad4036-9608-4b57-b812-97bf6c80c6de None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:25.380522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad4036-9608-4b57-b812-97bf6c80c6de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:25.381091 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad4036-9608-4b57-b812-97bf6c80c6de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:25.382553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad4036-9608-4b57-b812-97bf6c80c6de None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:25.395970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 639/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:34:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:25.402331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1492e-c5c2-4360-b0d1-3610a778f589 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:25.511271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1492e-c5c2-4360-b0d1-3610a778f589 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:25.547189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335b06d-dd73-4adc-8b63-ce32fb4eb37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:25.547937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b06d-dd73-4adc-8b63-ce32fb4eb37b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:25.548130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b06d-dd73-4adc-8b63-ce32fb4eb37b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:25.548398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b06d-dd73-4adc-8b63-ce32fb4eb37b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:25.549117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335b06d-dd73-4adc-8b63-ce32fb4eb37b None admin] RBAC: Adding query filter params (role.id=f0926be4d3d4462fac854212e101ccdd, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=fb770d4379e74bf79ad4280329c771c6) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:25.549415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335b06d-dd73-4adc-8b63-ce32fb4eb37b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:25.550857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335b06d-dd73-4adc-8b63-ce32fb4eb37b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:25.556649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 607/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:34:25 2022] GET /identity/v3/role_assignments?user.id=fb770d4379e74bf79ad4280329c771c6&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=f0926be4d3d4462fac854212e101ccdd => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:27.573687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56aa43-23eb-4006-977c-225efc808a7b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:27.574249 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56aa43-23eb-4006-977c-225efc808a7b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:27.574249 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56aa43-23eb-4006-977c-225efc808a7b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:27.575604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 640/1247] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:34:27.581399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2966cc-8583-449c-931b-66ab212ec3f8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:27.582023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2966cc-8583-449c-931b-66ab212ec3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:27.582023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2966cc-8583-449c-931b-66ab212ec3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:27.588530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:27.602448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-ca2966cc-8583-449c-931b-66ab212ec3f8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:34:27.603635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ca2966cc-8583-449c-931b-66ab212ec3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:34:27.672527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 608/1248] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:34:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:34:27.946628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56aa43-23eb-4006-977c-225efc808a7b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:27.965205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:28.049162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56aa43-23eb-4006-977c-225efc808a7b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:28.086645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800824-a5fc-43c5-bf20-03dbf90c51b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:28.087273 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800824-a5fc-43c5-bf20-03dbf90c51b3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:28.087546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800824-a5fc-43c5-bf20-03dbf90c51b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:28.087813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800824-a5fc-43c5-bf20-03dbf90c51b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:28.098095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4800824-a5fc-43c5-bf20-03dbf90c51b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e4800824-a5fc-43c5-bf20-03dbf90c51b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:34:28.103202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:28.104337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:34:28.105222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:28.106730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 641/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:34:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:28.111312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2966cc-8583-449c-931b-66ab212ec3f8 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:28.214314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2966cc-8583-449c-931b-66ab212ec3f8 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:28.244933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:28.250442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39672c91-9df6-4994-90cb-0fe221da60eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:28.251063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39672c91-9df6-4994-90cb-0fe221da60eb None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:28.251341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39672c91-9df6-4994-90cb-0fe221da60eb None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:28.251609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39672c91-9df6-4994-90cb-0fe221da60eb None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:28.252272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39672c91-9df6-4994-90cb-0fe221da60eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:28.252574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39672c91-9df6-4994-90cb-0fe221da60eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:28.253945 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39672c91-9df6-4994-90cb-0fe221da60eb None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:28.263078 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 609/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:34:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:28.270067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800824-a5fc-43c5-bf20-03dbf90c51b3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:28.351518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:28.375576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800824-a5fc-43c5-bf20-03dbf90c51b3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:28.410610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e6a5e-f799-41d6-be69-921f447b76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:28.411244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6a5e-f799-41d6-be69-921f447b76c9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:28.411519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6a5e-f799-41d6-be69-921f447b76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:28.411788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6a5e-f799-41d6-be69-921f447b76c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:28.422645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e6a5e-f799-41d6-be69-921f447b76c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:28.424156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e6a5e-f799-41d6-be69-921f447b76c9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-4e2e6a5e-f799-41d6-be69-921f447b76c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:28.439322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:28.440539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:28.441972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:28.441972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:28.441972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:28.441972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 642/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:28.446414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39672c91-9df6-4994-90cb-0fe221da60eb None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:28.551643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39672c91-9df6-4994-90cb-0fe221da60eb None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:28.586524 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b37daf-c2b4-4168-860c-75763a8bce90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:28.587173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b37daf-c2b4-4168-860c-75763a8bce90 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:28.587425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b37daf-c2b4-4168-860c-75763a8bce90 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:28.587699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b37daf-c2b4-4168-860c-75763a8bce90 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:28.588338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b37daf-c2b4-4168-860c-75763a8bce90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:28.588640 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b37daf-c2b4-4168-860c-75763a8bce90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:28.589995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b37daf-c2b4-4168-860c-75763a8bce90 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:28.602256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 610/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:28.608666 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e6a5e-f799-41d6-be69-921f447b76c9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:28.711409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e6a5e-f799-41d6-be69-921f447b76c9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:28.748568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:28.757055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b41c1-78d6-4839-a2cd-f6dd94711f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:28.757695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b41c1-78d6-4839-a2cd-f6dd94711f2f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:28.757961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b41c1-78d6-4839-a2cd-f6dd94711f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:28.758288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b41c1-78d6-4839-a2cd-f6dd94711f2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:28.767369 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b41c1-78d6-4839-a2cd-f6dd94711f2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:28.767564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b41c1-78d6-4839-a2cd-f6dd94711f2f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d67b41c1-78d6-4839-a2cd-f6dd94711f2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:34:28.778869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:28.780546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:28.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:28.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:28.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:34:28.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:34:28.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:34:28.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:34:28.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:34:28.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:34:28.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:28.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:28.782343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 643/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:28.785824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b37daf-c2b4-4168-860c-75763a8bce90 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:28.846339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:28.891400 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b37daf-c2b4-4168-860c-75763a8bce90 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:28.928481 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1236ab-5489-4f9a-8e3a-4a9cb9728c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:28.929104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1236ab-5489-4f9a-8e3a-4a9cb9728c2b None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:28.929382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1236ab-5489-4f9a-8e3a-4a9cb9728c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:28.929650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1236ab-5489-4f9a-8e3a-4a9cb9728c2b None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:28.930557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1236ab-5489-4f9a-8e3a-4a9cb9728c2b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:28.930862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1236ab-5489-4f9a-8e3a-4a9cb9728c2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:28.932327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1236ab-5489-4f9a-8e3a-4a9cb9728c2b None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:28.953789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 611/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Aug 29 23:34:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:28.960448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b41c1-78d6-4839-a2cd-f6dd94711f2f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:29.064470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b41c1-78d6-4839-a2cd-f6dd94711f2f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:29.098933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd5767-b203-4f03-963e-f5dcdcb0d72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:29.099444 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd5767-b203-4f03-963e-f5dcdcb0d72f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:29.099716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd5767-b203-4f03-963e-f5dcdcb0d72f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:29.099983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd5767-b203-4f03-963e-f5dcdcb0d72f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:29.156093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:29.168304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd5767-b203-4f03-963e-f5dcdcb0d72f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:29.170472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd5767-b203-4f03-963e-f5dcdcb0d72f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:29.270552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 644/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:34:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:34:29.276734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1236ab-5489-4f9a-8e3a-4a9cb9728c2b None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:29.382659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1236ab-5489-4f9a-8e3a-4a9cb9728c2b None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:29.397570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:29.419717 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516a4c8-b787-45db-890c-afda487b973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:29.420454 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516a4c8-b787-45db-890c-afda487b973f None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:29.420735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516a4c8-b787-45db-890c-afda487b973f None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:29.421003 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516a4c8-b787-45db-890c-afda487b973f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:29.430903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516a4c8-b787-45db-890c-afda487b973f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:29.432339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516a4c8-b787-45db-890c-afda487b973f None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-2516a4c8-b787-45db-890c-afda487b973f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:29.446752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:29.447959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:29.448917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:29.448917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:29.448917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:29.448917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 612/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:29.454141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd5767-b203-4f03-963e-f5dcdcb0d72f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:29.543145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:29.555837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd5767-b203-4f03-963e-f5dcdcb0d72f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:29.590528 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cd650-5c20-4653-9bc8-b6ddc53a60c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:29.591135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cd650-5c20-4653-9bc8-b6ddc53a60c7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:29.591419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cd650-5c20-4653-9bc8-b6ddc53a60c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:29.591690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cd650-5c20-4653-9bc8-b6ddc53a60c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:29.592325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cd650-5c20-4653-9bc8-b6ddc53a60c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:29.592622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cd650-5c20-4653-9bc8-b6ddc53a60c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:29.594082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cd650-5c20-4653-9bc8-b6ddc53a60c7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:29.605166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 645/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:29.611721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516a4c8-b787-45db-890c-afda487b973f None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:29.713836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516a4c8-b787-45db-890c-afda487b973f None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:29.749577 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67522203-6404-46d3-b23a-2b50db7096fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:29.750203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67522203-6404-46d3-b23a-2b50db7096fb None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:29.750534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67522203-6404-46d3-b23a-2b50db7096fb None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:29.750847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67522203-6404-46d3-b23a-2b50db7096fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:29.757983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67522203-6404-46d3-b23a-2b50db7096fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:29.759473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67522203-6404-46d3-b23a-2b50db7096fb None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-67522203-6404-46d3-b23a-2b50db7096fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:34:29.771050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:29.772162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:29.773464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:34:29.773464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:34:29.773464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:34:29.773464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:34:29.773464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:34:29.773464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:34:29.773464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:29.773464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:29.773464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 613/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:34:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:29.778033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cd650-5c20-4653-9bc8-b6ddc53a60c7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:29.887601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cd650-5c20-4653-9bc8-b6ddc53a60c7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:29.926480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670711dd-25b2-414f-8762-5fde76f9a426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:29.927089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670711dd-25b2-414f-8762-5fde76f9a426 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:29.927368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670711dd-25b2-414f-8762-5fde76f9a426 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:29.927638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670711dd-25b2-414f-8762-5fde76f9a426 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:29.928418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670711dd-25b2-414f-8762-5fde76f9a426 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:29.928735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670711dd-25b2-414f-8762-5fde76f9a426 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:29.930207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670711dd-25b2-414f-8762-5fde76f9a426 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:29.942665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 646/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:34:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:29.949072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67522203-6404-46d3-b23a-2b50db7096fb None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:29.976560 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:30.053588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67522203-6404-46d3-b23a-2b50db7096fb None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:30.088354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1342cf-15c9-4347-996f-e13abd543975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:30.088960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1342cf-15c9-4347-996f-e13abd543975 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:30.089239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1342cf-15c9-4347-996f-e13abd543975 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:30.089509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1342cf-15c9-4347-996f-e13abd543975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:30.090230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1342cf-15c9-4347-996f-e13abd543975 None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=fb770d4379e74bf79ad4280329c771c6) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:30.090569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1342cf-15c9-4347-996f-e13abd543975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:30.092597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1342cf-15c9-4347-996f-e13abd543975 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:30.098123 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 614/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:34:29 2022] GET /identity/v3/role_assignments?user.id=fb770d4379e74bf79ad4280329c771c6&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=d86dba8efe20423abdde06031690beb8 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:32.071716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477d66-aa3b-4ca3-a82b-f1325b397002 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:32.072250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477d66-aa3b-4ca3-a82b-f1325b397002 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:32.072250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477d66-aa3b-4ca3-a82b-f1325b397002 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:32.073613 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 647/1261] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:34:32.078962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcc890-ebd1-4162-82d2-5a408b68f3aa None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:32.079470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcc890-ebd1-4162-82d2-5a408b68f3aa None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:32.079564 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcc890-ebd1-4162-82d2-5a408b68f3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:32.099435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f8bcc890-ebd1-4162-82d2-5a408b68f3aa None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:34:32.100672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f8bcc890-ebd1-4162-82d2-5a408b68f3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:34:32.168304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 615/1262] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:34:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:34:32.441259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04477d66-aa3b-4ca3-a82b-f1325b397002 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:32.452793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:32.544474 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04477d66-aa3b-4ca3-a82b-f1325b397002 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:32.581791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d17ff-51dc-4d08-8293-6a3266e1e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:32.582415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d17ff-51dc-4d08-8293-6a3266e1e36a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:32.582716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d17ff-51dc-4d08-8293-6a3266e1e36a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:32.582997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d17ff-51dc-4d08-8293-6a3266e1e36a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:32.595687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d17ff-51dc-4d08-8293-6a3266e1e36a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-9a3d17ff-51dc-4d08-8293-6a3266e1e36a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:34:32.600797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:32.602083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:32.603076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 648/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:34:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:32.608683 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcc890-ebd1-4162-82d2-5a408b68f3aa None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:32.628620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:32.711336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcc890-ebd1-4162-82d2-5a408b68f3aa None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:32.748111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb87c97-fa19-4a87-9fad-9228923dd824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:32.748832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb87c97-fa19-4a87-9fad-9228923dd824 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:32.749105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb87c97-fa19-4a87-9fad-9228923dd824 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:32.749366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb87c97-fa19-4a87-9fad-9228923dd824 None admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:32.750031 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb87c97-fa19-4a87-9fad-9228923dd824 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:32.750328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb87c97-fa19-4a87-9fad-9228923dd824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:32.751770 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb87c97-fa19-4a87-9fad-9228923dd824 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:32.760844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 616/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:34:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:32.768346 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d17ff-51dc-4d08-8293-6a3266e1e36a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:32.840118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:32.875351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d17ff-51dc-4d08-8293-6a3266e1e36a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:32.911378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb088a-0ed2-4caf-8583-baefde838f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:32.911999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb088a-0ed2-4caf-8583-baefde838f8c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:32.912275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb088a-0ed2-4caf-8583-baefde838f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:32.912545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb088a-0ed2-4caf-8583-baefde838f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:32.923363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb088a-0ed2-4caf-8583-baefde838f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:32.924803 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb088a-0ed2-4caf-8583-baefde838f8c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-b2eb088a-0ed2-4caf-8583-baefde838f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:32.940445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:32.941813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:32.942810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:32.942810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:32.942810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:32.942810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 649/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:32.947659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb87c97-fa19-4a87-9fad-9228923dd824 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:33.031033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:33.051977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb87c97-fa19-4a87-9fad-9228923dd824 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:33.092241 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f602b-8f8b-4423-b61f-598d87b53c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:33.092951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f602b-8f8b-4423-b61f-598d87b53c47 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:33.093126 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f602b-8f8b-4423-b61f-598d87b53c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:33.093399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f602b-8f8b-4423-b61f-598d87b53c47 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:33.094047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f602b-8f8b-4423-b61f-598d87b53c47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:33.094348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f602b-8f8b-4423-b61f-598d87b53c47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:33.095757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f602b-8f8b-4423-b61f-598d87b53c47 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:33.108757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 617/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:33.114779 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb088a-0ed2-4caf-8583-baefde838f8c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:33.217279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb088a-0ed2-4caf-8583-baefde838f8c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:33.237011 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:33.251255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9b2be-5736-4b3c-b8ef-60c96482a80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:33.251955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9b2be-5736-4b3c-b8ef-60c96482a80c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:33.252289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9b2be-5736-4b3c-b8ef-60c96482a80c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:33.252572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9b2be-5736-4b3c-b8ef-60c96482a80c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:33.260550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9b2be-5736-4b3c-b8ef-60c96482a80c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:33.261987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9b2be-5736-4b3c-b8ef-60c96482a80c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-5fd9b2be-5736-4b3c-b8ef-60c96482a80c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:34:33.273620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:33.274816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:33.275770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:33.275770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:33.275770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:34:33.275770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:34:33.275770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:34:33.275770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:34:33.275770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:34:33.275770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:34:33.275770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:33.275770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:33.275770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 650/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:33.281916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f602b-8f8b-4423-b61f-598d87b53c47 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:33.386792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f602b-8f8b-4423-b61f-598d87b53c47 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:33.422977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799071a-967c-442e-9d0b-85497a59aa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:33.423586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799071a-967c-442e-9d0b-85497a59aa89 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:33.423862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799071a-967c-442e-9d0b-85497a59aa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:33.424131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799071a-967c-442e-9d0b-85497a59aa89 None admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:33.424947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3799071a-967c-442e-9d0b-85497a59aa89 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:33.425246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3799071a-967c-442e-9d0b-85497a59aa89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:33.426716 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3799071a-967c-442e-9d0b-85497a59aa89 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:33.436899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:33.445185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 618/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Aug 29 23:34:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:33.452044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9b2be-5736-4b3c-b8ef-60c96482a80c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:33.554450 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9b2be-5736-4b3c-b8ef-60c96482a80c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:33.590081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6826f9-77f2-48ab-95de-d7b514021a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:33.590781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6826f9-77f2-48ab-95de-d7b514021a85 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:33.591097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6826f9-77f2-48ab-95de-d7b514021a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:33.591682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6826f9-77f2-48ab-95de-d7b514021a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:33.602268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6826f9-77f2-48ab-95de-d7b514021a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:33.603819 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6826f9-77f2-48ab-95de-d7b514021a85 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-5a6826f9-77f2-48ab-95de-d7b514021a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:33.618012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:33.619193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:33.620104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:33.620104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:33.620104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:33.620104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 651/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:33.627795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799071a-967c-442e-9d0b-85497a59aa89 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:33.729022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799071a-967c-442e-9d0b-85497a59aa89 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:33.764007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd7bc8-2837-4d92-9e2d-2a67fddd2be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:33.764604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd7bc8-2837-4d92-9e2d-2a67fddd2be1 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:33.764882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd7bc8-2837-4d92-9e2d-2a67fddd2be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:33.765150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd7bc8-2837-4d92-9e2d-2a67fddd2be1 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:33.765797 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd7bc8-2837-4d92-9e2d-2a67fddd2be1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:33.766097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd7bc8-2837-4d92-9e2d-2a67fddd2be1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:33.767503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd7bc8-2837-4d92-9e2d-2a67fddd2be1 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:33.779931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 619/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:33.787034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6826f9-77f2-48ab-95de-d7b514021a85 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:33.822498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:33.896999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6826f9-77f2-48ab-95de-d7b514021a85 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:33.931147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b2902-f2c6-4b42-bf8f-b198a42f551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:33.931767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b2902-f2c6-4b42-bf8f-b198a42f551c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:33.932042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b2902-f2c6-4b42-bf8f-b198a42f551c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:33.932310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b2902-f2c6-4b42-bf8f-b198a42f551c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:33.939273 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b2902-f2c6-4b42-bf8f-b198a42f551c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:33.940773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b2902-f2c6-4b42-bf8f-b198a42f551c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a23b2902-f2c6-4b42-bf8f-b198a42f551c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:34:33.952574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:33.953703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:33.954872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:34:33.954872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:34:33.954872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:34:33.954872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:34:33.954872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:34:33.954872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:34:33.954872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:33.954872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:33.954872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 652/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:34:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:34:33.959768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd7bc8-2837-4d92-9e2d-2a67fddd2be1 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:34.024299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:34.073722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd7bc8-2837-4d92-9e2d-2a67fddd2be1 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:34.109972 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6048a6c-c52e-4062-9ce0-473108ab7b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:34.110691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6048a6c-c52e-4062-9ce0-473108ab7b85 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:34.110946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6048a6c-c52e-4062-9ce0-473108ab7b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:34.111212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6048a6c-c52e-4062-9ce0-473108ab7b85 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:34.111906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6048a6c-c52e-4062-9ce0-473108ab7b85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:34.112205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6048a6c-c52e-4062-9ce0-473108ab7b85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:34.113713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6048a6c-c52e-4062-9ce0-473108ab7b85 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:34.127975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 620/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:34:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:34.134834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b2902-f2c6-4b42-bf8f-b198a42f551c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:34.235428 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:34.244426 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b2902-f2c6-4b42-bf8f-b198a42f551c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:34.282768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcf158-76ee-4585-8869-99f81d069615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:34.283555 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcf158-76ee-4585-8869-99f81d069615 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:34.283837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcf158-76ee-4585-8869-99f81d069615 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:34.284135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcf158-76ee-4585-8869-99f81d069615 None admin] PATH_INFO: `/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/fb770d4379e74bf79ad4280329c771c6/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:34.286840 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcf158-76ee-4585-8869-99f81d069615 None admin] RBAC: Authorizing `identity:create_grant(project_id=eca691bde82145308c973e19aa9d1cbb, user_id=fb770d4379e74bf79ad4280329c771c6, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:34.288946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcf158-76ee-4585-8869-99f81d069615 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:34.412248 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 653/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Aug 29 23:34:34 2022] PUT /identity/v3/projects/eca691bde82145308c973e19aa9d1cbb/users/fb770d4379e74bf79ad4280329c771c6/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:34:36.333884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ded39-1512-48ef-b719-ba7069ee014b None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:36.334414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ded39-1512-48ef-b719-ba7069ee014b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:36.334562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ded39-1512-48ef-b719-ba7069ee014b None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:36.335855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 621/1274] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:34:36.341805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6318905-b890-4d3c-bd1b-0e055c9965fc None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:36.342451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6318905-b890-4d3c-bd1b-0e055c9965fc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:36.342451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6318905-b890-4d3c-bd1b-0e055c9965fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:36.361191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c6318905-b890-4d3c-bd1b-0e055c9965fc None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:34:36.362377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c6318905-b890-4d3c-bd1b-0e055c9965fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:34:36.433101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 654/1275] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:34:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:34:36.702108 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ded39-1512-48ef-b719-ba7069ee014b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:36.807426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ded39-1512-48ef-b719-ba7069ee014b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:36.825890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:36.843134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a027c99-4aba-4d52-b16c-69a8567c0d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:36.843834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027c99-4aba-4d52-b16c-69a8567c0d75 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:36.844110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027c99-4aba-4d52-b16c-69a8567c0d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:36.844375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027c99-4aba-4d52-b16c-69a8567c0d75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:36.855592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a027c99-4aba-4d52-b16c-69a8567c0d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-5a027c99-4aba-4d52-b16c-69a8567c0d75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 23:34:36.860967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:36.862091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:36.863287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 622/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Aug 29 23:34:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:36.868061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6318905-b890-4d3c-bd1b-0e055c9965fc None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:36.952567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:36.970709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6318905-b890-4d3c-bd1b-0e055c9965fc None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:37.005524 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533b4fb-6a5f-4557-a1a7-c39ac6f1c874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:37.006120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b4fb-6a5f-4557-a1a7-c39ac6f1c874 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:37.006392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b4fb-6a5f-4557-a1a7-c39ac6f1c874 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:37.006692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b4fb-6a5f-4557-a1a7-c39ac6f1c874 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:37.007341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533b4fb-6a5f-4557-a1a7-c39ac6f1c874 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:37.007637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533b4fb-6a5f-4557-a1a7-c39ac6f1c874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:37.009013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533b4fb-6a5f-4557-a1a7-c39ac6f1c874 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:37.017731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 655/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Aug 29 23:34:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:37.023513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a027c99-4aba-4d52-b16c-69a8567c0d75 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:37.130461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a027c99-4aba-4d52-b16c-69a8567c0d75 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:37.165222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe00f7e-ebb4-4515-9eb8-a9b13786d8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:37.165851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe00f7e-ebb4-4515-9eb8-a9b13786d8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:37.166129 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe00f7e-ebb4-4515-9eb8-a9b13786d8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:37.166399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe00f7e-ebb4-4515-9eb8-a9b13786d8a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:37.177035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe00f7e-ebb4-4515-9eb8-a9b13786d8a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:37.178500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe00f7e-ebb4-4515-9eb8-a9b13786d8a9 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-0fe00f7e-ebb4-4515-9eb8-a9b13786d8a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:37.193324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:37.199832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:37.201161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:37.201161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:37.201161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:37.201161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 623/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:37.201467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533b4fb-6a5f-4557-a1a7-c39ac6f1c874 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:37.307828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533b4fb-6a5f-4557-a1a7-c39ac6f1c874 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:37.343002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed1c59-5335-46b7-ad5a-eefc7b7d1362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:37.343693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1c59-5335-46b7-ad5a-eefc7b7d1362 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:37.343884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1c59-5335-46b7-ad5a-eefc7b7d1362 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:37.344159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1c59-5335-46b7-ad5a-eefc7b7d1362 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:37.344819 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed1c59-5335-46b7-ad5a-eefc7b7d1362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:37.345128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed1c59-5335-46b7-ad5a-eefc7b7d1362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:37.346607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed1c59-5335-46b7-ad5a-eefc7b7d1362 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:37.360660 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 656/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:34:37.366769 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe00f7e-ebb4-4515-9eb8-a9b13786d8a9 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:37.402218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:37.474470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe00f7e-ebb4-4515-9eb8-a9b13786d8a9 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:37.510950 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86607fd-18fe-45cd-a5e3-2e0c1f37a1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:37.511615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86607fd-18fe-45cd-a5e3-2e0c1f37a1fd None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:37.511917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86607fd-18fe-45cd-a5e3-2e0c1f37a1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:37.512187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86607fd-18fe-45cd-a5e3-2e0c1f37a1fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:37.520279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86607fd-18fe-45cd-a5e3-2e0c1f37a1fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:37.521777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86607fd-18fe-45cd-a5e3-2e0c1f37a1fd None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-c86607fd-18fe-45cd-a5e3-2e0c1f37a1fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 23:34:37.533369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:37.539366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:37.540576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:37.540576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:37.540576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 23:34:37.540576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 23:34:37.540576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 23:34:37.540576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 23:34:37.540576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 23:34:37.540576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 23:34:37.540576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 23:34:37.540576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:37.540576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 624/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:37.541389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed1c59-5335-46b7-ad5a-eefc7b7d1362 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:37.545390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:37.648250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed1c59-5335-46b7-ad5a-eefc7b7d1362 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:37.684205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc4700-3fc4-4d3e-aff0-8a46222bc132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:37.684949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc4700-3fc4-4d3e-aff0-8a46222bc132 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:37.685235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc4700-3fc4-4d3e-aff0-8a46222bc132 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:37.685511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc4700-3fc4-4d3e-aff0-8a46222bc132 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:37.686248 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc4700-3fc4-4d3e-aff0-8a46222bc132 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:37.686587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc4700-3fc4-4d3e-aff0-8a46222bc132 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:37.688055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc4700-3fc4-4d3e-aff0-8a46222bc132 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:37.706050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 657/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Aug 29 23:34:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:37.712329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86607fd-18fe-45cd-a5e3-2e0c1f37a1fd None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:37.804841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:37.815035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86607fd-18fe-45cd-a5e3-2e0c1f37a1fd None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:37.850329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebdfc1-255c-4e43-bd47-2c5c6e61df1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:37.850974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebdfc1-255c-4e43-bd47-2c5c6e61df1a None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:37.851247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebdfc1-255c-4e43-bd47-2c5c6e61df1a None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:37.851516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebdfc1-255c-4e43-bd47-2c5c6e61df1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:37.927946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebdfc1-255c-4e43-bd47-2c5c6e61df1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:37.929371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebdfc1-255c-4e43-bd47-2c5c6e61df1a None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:38.020758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:38.031072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 625/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Aug 29 23:34:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 23:34:38.037454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc4700-3fc4-4d3e-aff0-8a46222bc132 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:38.108184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:38.150503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc4700-3fc4-4d3e-aff0-8a46222bc132 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:38.188011 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6221a-f69a-4014-907a-1bae24a08646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:38.188861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6221a-f69a-4014-907a-1bae24a08646 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:38.189276 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6221a-f69a-4014-907a-1bae24a08646 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:38.189681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6221a-f69a-4014-907a-1bae24a08646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:38.203374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6221a-f69a-4014-907a-1bae24a08646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:38.204859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6221a-f69a-4014-907a-1bae24a08646 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-4aa6221a-f69a-4014-907a-1bae24a08646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 23:34:38.222188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:38.223488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 23:34:38.224573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 23:34:38.224573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 23:34:38.224573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:38.224573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 658/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:34:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:38.229352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebdfc1-255c-4e43-bd47-2c5c6e61df1a None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:38.340230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebdfc1-255c-4e43-bd47-2c5c6e61df1a None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:38.376307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34b117-bba4-4247-8b0d-54eb0d772e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:38.376968 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34b117-bba4-4247-8b0d-54eb0d772e53 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:38.377168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34b117-bba4-4247-8b0d-54eb0d772e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:38.377434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34b117-bba4-4247-8b0d-54eb0d772e53 None admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:38.378067 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34b117-bba4-4247-8b0d-54eb0d772e53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:38.378366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34b117-bba4-4247-8b0d-54eb0d772e53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:38.379853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34b117-bba4-4247-8b0d-54eb0d772e53 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:38.392152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 626/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Aug 29 23:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:38.397964 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6221a-f69a-4014-907a-1bae24a08646 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:38.510256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6221a-f69a-4014-907a-1bae24a08646 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:38.522885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:38.543833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45184b2-5492-47fc-9173-017c0c8e719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:38.544672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45184b2-5492-47fc-9173-017c0c8e719e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:38.544949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45184b2-5492-47fc-9173-017c0c8e719e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:38.545220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45184b2-5492-47fc-9173-017c0c8e719e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:38.551906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45184b2-5492-47fc-9173-017c0c8e719e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:38.553368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45184b2-5492-47fc-9173-017c0c8e719e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e45184b2-5492-47fc-9173-017c0c8e719e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 23:34:38.564648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:38.565788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:38.567007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:34:38.567007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:34:38.567007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:34:38.567007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:34:38.567007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:34:38.567007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:34:38.567007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 23:34:38.567007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 29 23:34:38.567007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 659/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Aug 29 23:34:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:38.573641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34b117-bba4-4247-8b0d-54eb0d772e53 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:38.626134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:38.676583 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34b117-bba4-4247-8b0d-54eb0d772e53 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:38.710488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a23d95-bba4-4c0f-b2ba-63e19eb9fa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:38.718856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a23d95-bba4-4c0f-b2ba-63e19eb9fa60 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:38.718856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a23d95-bba4-4c0f-b2ba-63e19eb9fa60 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:38.718856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a23d95-bba4-4c0f-b2ba-63e19eb9fa60 None admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:38.718856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a23d95-bba4-4c0f-b2ba-63e19eb9fa60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:38.718856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a23d95-bba4-4c0f-b2ba-63e19eb9fa60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:38.718856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a23d95-bba4-4c0f-b2ba-63e19eb9fa60 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:38.726194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 627/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Aug 29 23:34:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:38.732760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45184b2-5492-47fc-9173-017c0c8e719e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:38.835933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45184b2-5492-47fc-9173-017c0c8e719e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:38.872132 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a261fa6-2669-409b-9247-f32183b5c1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:38.872734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261fa6-2669-409b-9247-f32183b5c1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:38.873054 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261fa6-2669-409b-9247-f32183b5c1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:38.873287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261fa6-2669-409b-9247-f32183b5c1f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:38.874013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a261fa6-2669-409b-9247-f32183b5c1f9 None admin] RBAC: Adding query filter params (role.id=d86dba8efe20423abdde06031690beb8, scope.project.id=eca691bde82145308c973e19aa9d1cbb, user.id=fb770d4379e74bf79ad4280329c771c6) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:38.874323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a261fa6-2669-409b-9247-f32183b5c1f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:38.875813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a261fa6-2669-409b-9247-f32183b5c1f9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:38.882203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 660/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Aug 29 23:34:38 2022] GET /identity/v3/role_assignments?user.id=fb770d4379e74bf79ad4280329c771c6&scope.project.id=eca691bde82145308c973e19aa9d1cbb&role.id=d86dba8efe20423abdde06031690beb8 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:40.894395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745cb6-f13e-4560-a71d-650b7a4e012a None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:40.895060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745cb6-f13e-4560-a71d-650b7a4e012a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:40.895060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745cb6-f13e-4560-a71d-650b7a4e012a None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:40.896506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 628/1288] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:34:40.902512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d6ad-c036-4168-9522-383606d14413 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:40.904168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d6ad-c036-4168-9522-383606d14413 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:40.904168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d6ad-c036-4168-9522-383606d14413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:40.923334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6b76d6ad-c036-4168-9522-383606d14413 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:34:40.924606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6b76d6ad-c036-4168-9522-383606d14413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:34:40.955884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:40.993590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 661/1289] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:34:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:34:41.264867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73745cb6-f13e-4560-a71d-650b7a4e012a None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:41.367468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73745cb6-f13e-4560-a71d-650b7a4e012a None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:41.386013 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:41.409959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b227b-69de-4652-8ddb-e1ab08656770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:41.410625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b227b-69de-4652-8ddb-e1ab08656770 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:41.410891 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b227b-69de-4652-8ddb-e1ab08656770 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:41.411165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b227b-69de-4652-8ddb-e1ab08656770 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:41.411749 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b227b-69de-4652-8ddb-e1ab08656770 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:41.413141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b227b-69de-4652-8ddb-e1ab08656770 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-040b227b-69de-4652-8ddb-e1ab08656770 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:41.422220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:41.423536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:41.424523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 629/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Aug 29 23:34:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:41.429338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76d6ad-c036-4168-9522-383606d14413 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:41.534895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76d6ad-c036-4168-9522-383606d14413 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:41.572402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e442902-ad1b-48dc-b75b-232d18148ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:41.573022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442902-ad1b-48dc-b75b-232d18148ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:41.573298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442902-ad1b-48dc-b75b-232d18148ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:41.573570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442902-ad1b-48dc-b75b-232d18148ac4 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:41.574209 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442902-ad1b-48dc-b75b-232d18148ac4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:41.574546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442902-ad1b-48dc-b75b-232d18148ac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:41.576729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442902-ad1b-48dc-b75b-232d18148ac4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:41.582265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 662/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Mon Aug 29 23:34:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:41.588212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b227b-69de-4652-8ddb-e1ab08656770 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:41.690125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b227b-69de-4652-8ddb-e1ab08656770 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:41.725152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c868a-a8d3-43ea-b38e-1f23fce574b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:41.725747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c868a-a8d3-43ea-b38e-1f23fce574b9 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:41.726029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c868a-a8d3-43ea-b38e-1f23fce574b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:41.726305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c868a-a8d3-43ea-b38e-1f23fce574b9 None admin] PATH_INFO: `/v3/services` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:41.726976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c868a-a8d3-43ea-b38e-1f23fce574b9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:41.727281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c868a-a8d3-43ea-b38e-1f23fce574b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:41.728652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c868a-a8d3-43ea-b38e-1f23fce574b9 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:41.733237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 630/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Mon Aug 29 23:34:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:43.669614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499f6f4-b9fe-4f54-9dec-beb687aabcbc None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:43.670234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499f6f4-b9fe-4f54-9dec-beb687aabcbc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:43.670234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499f6f4-b9fe-4f54-9dec-beb687aabcbc None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:43.672305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 663/1293] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:34:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:34:43.678155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c5608-015b-4912-b6cc-b88a626aa10b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:43.678701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c5608-015b-4912-b6cc-b88a626aa10b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:43.678701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c5608-015b-4912-b6cc-b88a626aa10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:43.698019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-e55c5608-015b-4912-b6cc-b88a626aa10b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:34:43.698996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e55c5608-015b-4912-b6cc-b88a626aa10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:34:43.729584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:43.770378 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 631/1294] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:34:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:34:44.051111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499f6f4-b9fe-4f54-9dec-beb687aabcbc None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:44.085620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:44.162111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499f6f4-b9fe-4f54-9dec-beb687aabcbc None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:44.197050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0bcc0-dfcf-47a9-883d-fca2190d9c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:44.197651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0bcc0-dfcf-47a9-883d-fca2190d9c09 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:44.197930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0bcc0-dfcf-47a9-883d-fca2190d9c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:44.198201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0bcc0-dfcf-47a9-883d-fca2190d9c09 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:44.198886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0bcc0-dfcf-47a9-883d-fca2190d9c09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:44.200856 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0bcc0-dfcf-47a9-883d-fca2190d9c09 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:44.400619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 664/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Aug 29 23:34:44 2022] POST /identity/v3/services => generated 246 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:34:46.410035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5ed20-30dd-4803-b978-d82ab32799ce None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:46.410653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5ed20-30dd-4803-b978-d82ab32799ce None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:46.410653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5ed20-30dd-4803-b978-d82ab32799ce None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:46.412003 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 632/1296] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:34:46.420046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d8449-515e-4bf1-be9f-a9d30541991e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:46.420677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d8449-515e-4bf1-be9f-a9d30541991e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:46.420677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d8449-515e-4bf1-be9f-a9d30541991e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:46.441214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-577d8449-515e-4bf1-be9f-a9d30541991e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:34:46.445545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-577d8449-515e-4bf1-be9f-a9d30541991e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:34:46.552340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 665/1297] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:34:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:34:46.822155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5ed20-30dd-4803-b978-d82ab32799ce None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:46.934526 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5ed20-30dd-4803-b978-d82ab32799ce None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:46.970067 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c1923-c4c2-4a44-b3b1-da99ad2ec580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:46.970728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c1923-c4c2-4a44-b3b1-da99ad2ec580 None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:46.970998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c1923-c4c2-4a44-b3b1-da99ad2ec580 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:46.971262 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c1923-c4c2-4a44-b3b1-da99ad2ec580 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:46.971841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c1923-c4c2-4a44-b3b1-da99ad2ec580 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:46.973349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c1923-c4c2-4a44-b3b1-da99ad2ec580 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-cd0c1923-c4c2-4a44-b3b1-da99ad2ec580 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:46.982927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:46.984148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:46.985209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 633/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Aug 29 23:34:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 23:34:46.990200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d8449-515e-4bf1-be9f-a9d30541991e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:47.009197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:47.100573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d8449-515e-4bf1-be9f-a9d30541991e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:47.134350 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceffc6d-c567-4dbb-ad2d-b296be2bf2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:47.134988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceffc6d-c567-4dbb-ad2d-b296be2bf2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:47.135266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceffc6d-c567-4dbb-ad2d-b296be2bf2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:47.135541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceffc6d-c567-4dbb-ad2d-b296be2bf2d7 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:47.136168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceffc6d-c567-4dbb-ad2d-b296be2bf2d7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:47.136467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceffc6d-c567-4dbb-ad2d-b296be2bf2d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:47.138676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceffc6d-c567-4dbb-ad2d-b296be2bf2d7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:47.143884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 666/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Mon Aug 29 23:34:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:47.150194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c1923-c4c2-4a44-b3b1-da99ad2ec580 None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:47.183904 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:47.252668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c1923-c4c2-4a44-b3b1-da99ad2ec580 None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:47.286903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffece73f-22d4-429d-9085-bacdc8f8810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:47.287499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece73f-22d4-429d-9085-bacdc8f8810e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:47.287775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece73f-22d4-429d-9085-bacdc8f8810e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:47.288039 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece73f-22d4-429d-9085-bacdc8f8810e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:47.288728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffece73f-22d4-429d-9085-bacdc8f8810e None admin] RBAC: Adding query filter params (interface=public, service_id=4487bfe20cfb45bfb98c69938d9f256d, region_id=RegionOne) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:47.289033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffece73f-22d4-429d-9085-bacdc8f8810e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:47.290385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffece73f-22d4-429d-9085-bacdc8f8810e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:47.295522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 634/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Aug 29 23:34:47 2022] GET /identity/v3/endpoints?service_id=4487bfe20cfb45bfb98c69938d9f256d&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:47.301381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceffc6d-c567-4dbb-ad2d-b296be2bf2d7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:47.385911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:47.406593 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceffc6d-c567-4dbb-ad2d-b296be2bf2d7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:47.442187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fbedb-4f02-4eaf-a2fa-3c419cf7c080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:47.442897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fbedb-4f02-4eaf-a2fa-3c419cf7c080 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:47.443071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fbedb-4f02-4eaf-a2fa-3c419cf7c080 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:47.443429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fbedb-4f02-4eaf-a2fa-3c419cf7c080 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:47.444104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fbedb-4f02-4eaf-a2fa-3c419cf7c080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:47.445528 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fbedb-4f02-4eaf-a2fa-3c419cf7c080 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:47.453281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 667/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Aug 29 23:34:47 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:34:49.397442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ddfb8-d366-420a-9e6e-e61c5f533456 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:49.398000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ddfb8-d366-420a-9e6e-e61c5f533456 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:49.398000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ddfb8-d366-420a-9e6e-e61c5f533456 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:49.399379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 635/1302] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:34:49.404928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0a02-0b41-4635-bfef-094578ac6880 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:49.405407 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0a02-0b41-4635-bfef-094578ac6880 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:49.405509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0a02-0b41-4635-bfef-094578ac6880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:49.424210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-30bc0a02-0b41-4635-bfef-094578ac6880 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:34:49.425425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-30bc0a02-0b41-4635-bfef-094578ac6880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:34:49.494512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 668/1303] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:34:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 23:34:49.763495 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ddfb8-d366-420a-9e6e-e61c5f533456 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:49.879774 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:49.886540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ddfb8-d366-420a-9e6e-e61c5f533456 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:49.922453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a4943-f412-4e76-abd3-3d04552e0d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:49.923127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a4943-f412-4e76-abd3-3d04552e0d7e None admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:49.923399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a4943-f412-4e76-abd3-3d04552e0d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:49.923674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a4943-f412-4e76-abd3-3d04552e0d7e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:49.924263 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a4943-f412-4e76-abd3-3d04552e0d7e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:49.925628 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a4943-f412-4e76-abd3-3d04552e0d7e None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application [None req-407a4943-f412-4e76-abd3-3d04552e0d7e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:34:49.935033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:34:49.936219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application with Lock( Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: ERROR keystone.server.flask.application Aug 29 23:34:49.937185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 636/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Aug 29 23:34:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 23:34:49.942328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc0a02-0b41-4635-bfef-094578ac6880 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:50.021828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:34:50.044515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc0a02-0b41-4635-bfef-094578ac6880 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:50.081070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2c9bf-7db5-4bb8-ab68-1558d71f53c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:50.081688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c9bf-7db5-4bb8-ab68-1558d71f53c1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:50.081965 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c9bf-7db5-4bb8-ab68-1558d71f53c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:50.082236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c9bf-7db5-4bb8-ab68-1558d71f53c1 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:50.082897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2c9bf-7db5-4bb8-ab68-1558d71f53c1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:34:50.083199 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2c9bf-7db5-4bb8-ab68-1558d71f53c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:50.084571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2c9bf-7db5-4bb8-ab68-1558d71f53c1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:50.091005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 669/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Mon Aug 29 23:34:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:34:50.096864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a4943-f412-4e76-abd3-3d04552e0d7e None admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:34:50.207488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a4943-f412-4e76-abd3-3d04552e0d7e None admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:34:50.242093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97481b-0adf-4ff7-88ae-db3369c6e225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:34:50.242741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97481b-0adf-4ff7-88ae-db3369c6e225 None admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:34:50.243017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97481b-0adf-4ff7-88ae-db3369c6e225 None admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:34:50.243283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97481b-0adf-4ff7-88ae-db3369c6e225 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:34:50.243944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97481b-0adf-4ff7-88ae-db3369c6e225 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:34:50.245301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97481b-0adf-4ff7-88ae-db3369c6e225 None admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:34:50.365629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 637/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:34:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:35:00.087009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=30 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:35:00.360745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:34.179310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a580153-d603-4f5f-8fb3-1c46e8123f32 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:34.180302 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a580153-d603-4f5f-8fb3-1c46e8123f32 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:34.180798 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a580153-d603-4f5f-8fb3-1c46e8123f32 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:34.187737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de2aee-f274-4429-a124-3179ed158418 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:34.188724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de2aee-f274-4429-a124-3179ed158418 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:34.189221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de2aee-f274-4429-a124-3179ed158418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:34.192383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 670/1307] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:42:34 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:42:34.281264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-c4de2aee-f274-4429-a124-3179ed158418 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:42:34.283611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c4de2aee-f274-4429-a124-3179ed158418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:34.551729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 638/1308] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:42:34.825781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ddc92-7341-4390-8b7c-beb821f9a5cb None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:34.826062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ddc92-7341-4390-8b7c-beb821f9a5cb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:34.826650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ddc92-7341-4390-8b7c-beb821f9a5cb None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:34.828061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 671/1309] 10.222.0.58 () {54 vars in 965 bytes} [Mon Aug 29 23:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:42:34.834637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a0859-9c60-42e1-bcfd-e445c7c49808 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:34.835003 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a0859-9c60-42e1-bcfd-e445c7c49808 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:34.835384 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a0859-9c60-42e1-bcfd-e445c7c49808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:34.921098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-239a0859-9c60-42e1-bcfd-e445c7c49808 None None] MFA Rules not processed for user `863bcfa6b8694182b9d163133a141907`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:42:34.922478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-239a0859-9c60-42e1-bcfd-e445c7c49808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:34.948999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:35.012599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 639/1310] 10.222.0.58 () {58 vars in 1062 bytes} [Mon Aug 29 23:42:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:42:35.022804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ddc92-7341-4390-8b7c-beb821f9a5cb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:42:35.024616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5e2ddc92-7341-4390-8b7c-beb821f9a5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:35.149860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ddc92-7341-4390-8b7c-beb821f9a5cb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:42:35.226979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a8ffe-ce7f-47ff-bbab-57bd69060e42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:42:35.227766 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8ffe-ce7f-47ff-bbab-57bd69060e42 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:35.227856 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8ffe-ce7f-47ff-bbab-57bd69060e42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:35.228153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8ffe-ce7f-47ff-bbab-57bd69060e42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:35.229699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a60a8ffe-ce7f-47ff-bbab-57bd69060e42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:35.288664 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:35.350676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a8ffe-ce7f-47ff-bbab-57bd69060e42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:42:35.352536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a8ffe-ce7f-47ff-bbab-57bd69060e42 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:42:35.372520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 672/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:42:37.603486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968fb06-d6c6-430b-b0d9-3b3485fbb558 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:37.603978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968fb06-d6c6-430b-b0d9-3b3485fbb558 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:37.603978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968fb06-d6c6-430b-b0d9-3b3485fbb558 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:37.605458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 640/1312] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:42:37.610488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e9a9-43ca-4790-9126-429188d8ef2f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:37.610962 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e9a9-43ca-4790-9126-429188d8ef2f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:37.611339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e9a9-43ca-4790-9126-429188d8ef2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:37.687931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6728e9a9-43ca-4790-9126-429188d8ef2f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:42:37.689672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6728e9a9-43ca-4790-9126-429188d8ef2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:37.796816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 673/1313] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:42:38.107718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42226d7-0c2a-4887-a644-2d07c7dbde9c None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:38.107996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42226d7-0c2a-4887-a644-2d07c7dbde9c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:38.108267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42226d7-0c2a-4887-a644-2d07c7dbde9c None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:38.109626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 641/1314] 10.222.0.58 () {54 vars in 965 bytes} [Mon Aug 29 23:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:42:38.116527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99823bd-fcf8-4fc1-b984-d42591f2331b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:38.116812 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99823bd-fcf8-4fc1-b984-d42591f2331b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:38.117086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99823bd-fcf8-4fc1-b984-d42591f2331b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:38.131289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:38.199445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c99823bd-fcf8-4fc1-b984-d42591f2331b None None] MFA Rules not processed for user `863bcfa6b8694182b9d163133a141907`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:42:38.200648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c99823bd-fcf8-4fc1-b984-d42591f2331b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:38.279945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 674/1315] 10.222.0.58 () {58 vars in 1062 bytes} [Mon Aug 29 23:42:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:42:38.289944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42226d7-0c2a-4887-a644-2d07c7dbde9c None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:42:38.291448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a42226d7-0c2a-4887-a644-2d07c7dbde9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:38.401838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42226d7-0c2a-4887-a644-2d07c7dbde9c None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:42:38.481144 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:38.483789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d861bc-a4ff-4e27-b75e-45081dc2a62d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:42:38.484412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d861bc-a4ff-4e27-b75e-45081dc2a62d service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:38.484690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d861bc-a4ff-4e27-b75e-45081dc2a62d service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:38.484964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d861bc-a4ff-4e27-b75e-45081dc2a62d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:38.487079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-33d861bc-a4ff-4e27-b75e-45081dc2a62d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:38.611527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d861bc-a4ff-4e27-b75e-45081dc2a62d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:42:38.612913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d861bc-a4ff-4e27-b75e-45081dc2a62d service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:42:38.633559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 642/1316] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:42:41.392037 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313e5bc-550c-4508-9acb-f551835c6f8d None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:41.392648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313e5bc-550c-4508-9acb-f551835c6f8d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:41.392880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313e5bc-550c-4508-9acb-f551835c6f8d None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:41.394374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 675/1317] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:42:41.399541 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92858038-25f9-4f5a-a7e9-c6cb583dc098 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:41.399930 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92858038-25f9-4f5a-a7e9-c6cb583dc098 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:41.400083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92858038-25f9-4f5a-a7e9-c6cb583dc098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:41.466561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-92858038-25f9-4f5a-a7e9-c6cb583dc098 None None] MFA Rules not processed for user `3d90934d388e445184cf04273effcac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:42:41.467772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-92858038-25f9-4f5a-a7e9-c6cb583dc098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:41.520225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:41.563737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 643/1318] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:42:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:42:41.849065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313e5bc-550c-4508-9acb-f551835c6f8d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:42:41.850645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7313e5bc-550c-4508-9acb-f551835c6f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:41.953466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313e5bc-550c-4508-9acb-f551835c6f8d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:42:41.956844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:42.028356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030f67b-3e15-417e-a6aa-dde59e732f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:42:42.028848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030f67b-3e15-417e-a6aa-dde59e732f38 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:42.029122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030f67b-3e15-417e-a6aa-dde59e732f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:42.029391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030f67b-3e15-417e-a6aa-dde59e732f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:42.031001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4030f67b-3e15-417e-a6aa-dde59e732f38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:42.145134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030f67b-3e15-417e-a6aa-dde59e732f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:42:42.146609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030f67b-3e15-417e-a6aa-dde59e732f38 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:42:42.164968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 676/1319] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:42:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:42:45.003327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cce73-7249-423f-926f-f794d4eb7904 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:45.003867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cce73-7249-423f-926f-f794d4eb7904 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:45.003867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cce73-7249-423f-926f-f794d4eb7904 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:45.005240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 644/1320] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:42:45.010739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cef93-b10f-428e-a0a2-060bea01e746 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:45.011170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cef93-b10f-428e-a0a2-060bea01e746 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:45.011393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cef93-b10f-428e-a0a2-060bea01e746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:45.052884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:45.078443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-208cef93-b10f-428e-a0a2-060bea01e746 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:42:45.079665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-208cef93-b10f-428e-a0a2-060bea01e746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:45.177659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 677/1321] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:42:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:42:45.466949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cce73-7249-423f-926f-f794d4eb7904 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:42:45.468567 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-955cce73-7249-423f-926f-f794d4eb7904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:45.573611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cce73-7249-423f-926f-f794d4eb7904 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:42:45.646673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e5e14-bef7-44c6-be44-9b569865911d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:42:45.647093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e5e14-bef7-44c6-be44-9b569865911d service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:45.647373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e5e14-bef7-44c6-be44-9b569865911d service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:45.647644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e5e14-bef7-44c6-be44-9b569865911d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:45.649236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-065e5e14-bef7-44c6-be44-9b569865911d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:45.656549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:45.759942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e5e14-bef7-44c6-be44-9b569865911d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:42:45.761374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e5e14-bef7-44c6-be44-9b569865911d service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:42:45.779693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 645/1322] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:42:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:42:48.963247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eb307-2926-4da0-bbfc-eca736358f8a None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:48.963890 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eb307-2926-4da0-bbfc-eca736358f8a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:48.963890 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eb307-2926-4da0-bbfc-eca736358f8a None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:48.965135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 678/1323] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:42:48.971531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729545a7-812a-43e3-bbfc-142ebf9056c3 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:48.972045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729545a7-812a-43e3-bbfc-142ebf9056c3 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:48.972373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729545a7-812a-43e3-bbfc-142ebf9056c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:49.048389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-729545a7-812a-43e3-bbfc-142ebf9056c3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:42:49.049674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-729545a7-812a-43e3-bbfc-142ebf9056c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:49.111021 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:49.157704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 646/1324] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:42:49.444042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eb307-2926-4da0-bbfc-eca736358f8a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:42:49.445891 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-962eb307-2926-4da0-bbfc-eca736358f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:49.556207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eb307-2926-4da0-bbfc-eca736358f8a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:42:49.569568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:49.633026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd789f2-9bd6-4c77-9d01-71a838f8ddbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:42:49.633549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd789f2-9bd6-4c77-9d01-71a838f8ddbd service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:49.633828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd789f2-9bd6-4c77-9d01-71a838f8ddbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:49.634100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd789f2-9bd6-4c77-9d01-71a838f8ddbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:49.635734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3dd789f2-9bd6-4c77-9d01-71a838f8ddbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:49.747936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd789f2-9bd6-4c77-9d01-71a838f8ddbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:42:49.749314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd789f2-9bd6-4c77-9d01-71a838f8ddbd service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:42:49.770292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 679/1325] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:42:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:42:51.765198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c182c-49f4-46f8-af7f-64471b6f9af7 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:51.765837 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c182c-49f4-46f8-af7f-64471b6f9af7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:51.765837 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c182c-49f4-46f8-af7f-64471b6f9af7 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:51.768303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 647/1326] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:42:51.774041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c38bbc-8a53-4521-8483-82f091036855 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:51.774608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c38bbc-8a53-4521-8483-82f091036855 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:51.774709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c38bbc-8a53-4521-8483-82f091036855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:51.828002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:51.846499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d1c38bbc-8a53-4521-8483-82f091036855 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:42:51.847763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d1c38bbc-8a53-4521-8483-82f091036855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:51.948399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 680/1327] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:42:52.249867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c182c-49f4-46f8-af7f-64471b6f9af7 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:42:52.251420 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-521c182c-49f4-46f8-af7f-64471b6f9af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:52.368096 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c182c-49f4-46f8-af7f-64471b6f9af7 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:42:52.449102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782bfc2-84c6-4f89-851d-d0beced86012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:42:52.449723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782bfc2-84c6-4f89-851d-d0beced86012 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:52.450001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782bfc2-84c6-4f89-851d-d0beced86012 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:52.450271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782bfc2-84c6-4f89-851d-d0beced86012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:52.452081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4782bfc2-84c6-4f89-851d-d0beced86012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:52.472693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:52.588224 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782bfc2-84c6-4f89-851d-d0beced86012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:42:52.590094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782bfc2-84c6-4f89-851d-d0beced86012 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:42:52.609489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 648/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:42:54.631410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d46fb9-c840-43a3-ab59-ec9cbde608d8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:54.631933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d46fb9-c840-43a3-ab59-ec9cbde608d8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:54.631933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d46fb9-c840-43a3-ab59-ec9cbde608d8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:54.633229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 681/1329] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:42:54.638712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2a63b-fa23-4265-b8ce-73088a1239d2 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:54.639213 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2a63b-fa23-4265-b8ce-73088a1239d2 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:54.639397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2a63b-fa23-4265-b8ce-73088a1239d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:54.712428 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-3eb2a63b-fa23-4265-b8ce-73088a1239d2 None None] MFA Rules not processed for user `3d90934d388e445184cf04273effcac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:42:54.713639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3eb2a63b-fa23-4265-b8ce-73088a1239d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:54.784517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:54.818709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 649/1330] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:42:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:42:55.105410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d46fb9-c840-43a3-ab59-ec9cbde608d8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:42:55.106963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-81d46fb9-c840-43a3-ab59-ec9cbde608d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:55.214167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d46fb9-c840-43a3-ab59-ec9cbde608d8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:42:55.257552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:55.317874 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706011d5-affc-451e-8769-845d68690ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:42:55.318658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706011d5-affc-451e-8769-845d68690ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:55.318935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706011d5-affc-451e-8769-845d68690ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:55.319206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706011d5-affc-451e-8769-845d68690ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:55.320875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-706011d5-affc-451e-8769-845d68690ee2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:55.446007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706011d5-affc-451e-8769-845d68690ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:42:55.447765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706011d5-affc-451e-8769-845d68690ee2 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:42:55.466775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 682/1331] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:42:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:42:57.900632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ebbfa-f5bd-4c9c-98a9-43dbe3f19bc3 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:57.901275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ebbfa-f5bd-4c9c-98a9-43dbe3f19bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:57.901275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ebbfa-f5bd-4c9c-98a9-43dbe3f19bc3 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:57.902716 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 650/1332] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:42:57.909242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f43cc-e741-41a6-9aa0-4fe00ad2a7d4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:57.909775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f43cc-e741-41a6-9aa0-4fe00ad2a7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:57.910185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f43cc-e741-41a6-9aa0-4fe00ad2a7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:58.004660 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:58.008144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6c1f43cc-e741-41a6-9aa0-4fe00ad2a7d4 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:42:58.009381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6c1f43cc-e741-41a6-9aa0-4fe00ad2a7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:58.117201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 683/1333] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:42:58.435957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ebbfa-f5bd-4c9c-98a9-43dbe3f19bc3 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:42:58.437496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a92ebbfa-f5bd-4c9c-98a9-43dbe3f19bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:58.546620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ebbfa-f5bd-4c9c-98a9-43dbe3f19bc3 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:42:58.628898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402384d-9444-4ce0-999e-29643903a29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:42:58.629512 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402384d-9444-4ce0-999e-29643903a29d service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:42:58.629790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402384d-9444-4ce0-999e-29643903a29d service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:42:58.630058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402384d-9444-4ce0-999e-29643903a29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:42:58.631755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1402384d-9444-4ce0-999e-29643903a29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:42:58.663746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:42:58.753264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402384d-9444-4ce0-999e-29643903a29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:42:58.754608 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402384d-9444-4ce0-999e-29643903a29d service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:42:58.774529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 651/1334] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:43:00.809438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7a0d3-bbd9-4bad-a154-0a074e189c4c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:00.809990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7a0d3-bbd9-4bad-a154-0a074e189c4c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:00.809990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7a0d3-bbd9-4bad-a154-0a074e189c4c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:00.811320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 684/1335] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:00.817269 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475b8ad-5e75-4d22-a762-da09ded82b9c None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:00.817745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475b8ad-5e75-4d22-a762-da09ded82b9c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:00.817853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475b8ad-5e75-4d22-a762-da09ded82b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:00.885338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-a475b8ad-5e75-4d22-a762-da09ded82b9c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:00.886587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a475b8ad-5e75-4d22-a762-da09ded82b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:00.967086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:00.989962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 652/1336] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:01.286485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7a0d3-bbd9-4bad-a154-0a074e189c4c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:01.288092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2ad7a0d3-bbd9-4bad-a154-0a074e189c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:01.402385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7a0d3-bbd9-4bad-a154-0a074e189c4c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:01.433073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:01.484353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbed8e-3d7a-4b5f-8d2a-33707a4fd541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:01.484962 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbed8e-3d7a-4b5f-8d2a-33707a4fd541 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:01.485232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbed8e-3d7a-4b5f-8d2a-33707a4fd541 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:01.485492 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbed8e-3d7a-4b5f-8d2a-33707a4fd541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:01.487805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6efbed8e-3d7a-4b5f-8d2a-33707a4fd541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:01.604423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbed8e-3d7a-4b5f-8d2a-33707a4fd541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:01.605795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbed8e-3d7a-4b5f-8d2a-33707a4fd541 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:01.629962 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 685/1337] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:43:04.273457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd61f-43dc-409e-bac6-6ba20818fbc7 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:04.274064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd61f-43dc-409e-bac6-6ba20818fbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:04.274064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd61f-43dc-409e-bac6-6ba20818fbc7 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:04.275436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 653/1338] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:04.282129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65a71-225d-45d7-8269-2bc0774316cb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:04.282696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65a71-225d-45d7-8269-2bc0774316cb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:04.282696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65a71-225d-45d7-8269-2bc0774316cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:04.364209 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-50e65a71-225d-45d7-8269-2bc0774316cb None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:04.365425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-50e65a71-225d-45d7-8269-2bc0774316cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:04.374239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:04.471484 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 686/1339] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:04.760324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dd61f-43dc-409e-bac6-6ba20818fbc7 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:04.761871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2d4dd61f-43dc-409e-bac6-6ba20818fbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:04.880331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dd61f-43dc-409e-bac6-6ba20818fbc7 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:04.971199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369364cd-dc87-408c-a889-e084d0d61be0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:04.972518 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369364cd-dc87-408c-a889-e084d0d61be0 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:04.972682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369364cd-dc87-408c-a889-e084d0d61be0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:04.972957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369364cd-dc87-408c-a889-e084d0d61be0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:04.974655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-369364cd-dc87-408c-a889-e084d0d61be0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:05.013294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:05.092940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369364cd-dc87-408c-a889-e084d0d61be0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:05.094407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369364cd-dc87-408c-a889-e084d0d61be0 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:05.118153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 654/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:43:09.839960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5849c-77c0-475d-8134-87af921601b0 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:09.840548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5849c-77c0-475d-8134-87af921601b0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:09.840665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5849c-77c0-475d-8134-87af921601b0 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:09.842327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 687/1341] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:09.848058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c28ae-3f6a-4f11-8c45-d159e71c50a1 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:09.848681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c28ae-3f6a-4f11-8c45-d159e71c50a1 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:09.848681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c28ae-3f6a-4f11-8c45-d159e71c50a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:09.917732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-104c28ae-3f6a-4f11-8c45-d159e71c50a1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:09.919289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-104c28ae-3f6a-4f11-8c45-d159e71c50a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:10.016976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:10.029997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 655/1342] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:10.322135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5849c-77c0-475d-8134-87af921601b0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:10.324034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e5e5849c-77c0-475d-8134-87af921601b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:10.443583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5849c-77c0-475d-8134-87af921601b0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:10.481700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:10.520835 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a7606-e2a7-4d78-89c5-a2b5a806a106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:10.521808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7606-e2a7-4d78-89c5-a2b5a806a106 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:10.522174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7606-e2a7-4d78-89c5-a2b5a806a106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:10.522595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7606-e2a7-4d78-89c5-a2b5a806a106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:10.524287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-858a7606-e2a7-4d78-89c5-a2b5a806a106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:10.649394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a7606-e2a7-4d78-89c5-a2b5a806a106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:10.650897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a7606-e2a7-4d78-89c5-a2b5a806a106 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:10.672300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 688/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:43:13.889237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c8031-daf4-4f66-865f-46106dcf46b7 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:13.890112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c8031-daf4-4f66-865f-46106dcf46b7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:13.890393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c8031-daf4-4f66-865f-46106dcf46b7 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:13.891769 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 656/1344] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:13.897475 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231bbde-a9cd-4879-8b81-63857363fcb7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:13.898027 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231bbde-a9cd-4879-8b81-63857363fcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:13.898027 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231bbde-a9cd-4879-8b81-63857363fcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:13.967275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6231bbde-a9cd-4879-8b81-63857363fcb7 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:13.968485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6231bbde-a9cd-4879-8b81-63857363fcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:13.988627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:14.070321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 689/1345] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:14.360073 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c8031-daf4-4f66-865f-46106dcf46b7 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:14.360073 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bc8c8031-daf4-4f66-865f-46106dcf46b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:14.478867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c8031-daf4-4f66-865f-46106dcf46b7 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:14.558858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072814d4-4f3b-4843-b400-68d278047f16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:14.559733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072814d4-4f3b-4843-b400-68d278047f16 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:14.560007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072814d4-4f3b-4843-b400-68d278047f16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:14.560273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072814d4-4f3b-4843-b400-68d278047f16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:14.562141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-072814d4-4f3b-4843-b400-68d278047f16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:14.608333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:14.684270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072814d4-4f3b-4843-b400-68d278047f16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:14.685744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072814d4-4f3b-4843-b400-68d278047f16 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:14.707881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 657/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:43:19.035770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeef14e-9169-4545-b401-35d5d2a7d622 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:19.036350 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeef14e-9169-4545-b401-35d5d2a7d622 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:19.036456 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeef14e-9169-4545-b401-35d5d2a7d622 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:19.037776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 690/1347] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:19.043254 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365538a3-a4fd-4bd2-8cf2-c9736f2d9114 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:19.043743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365538a3-a4fd-4bd2-8cf2-c9736f2d9114 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:19.043836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365538a3-a4fd-4bd2-8cf2-c9736f2d9114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:19.137815 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-365538a3-a4fd-4bd2-8cf2-c9736f2d9114 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:19.139513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-365538a3-a4fd-4bd2-8cf2-c9736f2d9114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:19.249871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:19.252614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 658/1348] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:19.553871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeef14e-9169-4545-b401-35d5d2a7d622 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:19.555487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5aeef14e-9169-4545-b401-35d5d2a7d622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:19.693570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeef14e-9169-4545-b401-35d5d2a7d622 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:19.745343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:19.776722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9820e1-b18e-440b-b28e-152b19fb7ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:19.777349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9820e1-b18e-440b-b28e-152b19fb7ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:19.777626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9820e1-b18e-440b-b28e-152b19fb7ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:19.777893 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9820e1-b18e-440b-b28e-152b19fb7ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:19.779792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bf9820e1-b18e-440b-b28e-152b19fb7ce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:19.920370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9820e1-b18e-440b-b28e-152b19fb7ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:19.922396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9820e1-b18e-440b-b28e-152b19fb7ce9 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:19.944724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 691/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:43:25.028551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43386d-0e1e-446e-9d67-f012a835e36c None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:25.029370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43386d-0e1e-446e-9d67-f012a835e36c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:25.029803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43386d-0e1e-446e-9d67-f012a835e36c None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:25.031637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 659/1350] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:25.040756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d072ffe-bd10-4b3f-8b51-00a82a411bfa None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:25.042129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d072ffe-bd10-4b3f-8b51-00a82a411bfa None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:25.042412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d072ffe-bd10-4b3f-8b51-00a82a411bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:25.140969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6d072ffe-bd10-4b3f-8b51-00a82a411bfa None None] MFA Rules not processed for user `fb770d4379e74bf79ad4280329c771c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:25.142379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6d072ffe-bd10-4b3f-8b51-00a82a411bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:25.168204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:25.331651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 692/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:43:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:29.939051 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab6a52-dab5-411c-b597-0271882c0623 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:29.939615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab6a52-dab5-411c-b597-0271882c0623 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:29.939615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab6a52-dab5-411c-b597-0271882c0623 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:29.941167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 660/1352] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:29.946662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566ae62-9dda-49e1-8e41-68502db0bcd9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:29.947170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566ae62-9dda-49e1-8e41-68502db0bcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:29.947312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566ae62-9dda-49e1-8e41-68502db0bcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:30.032622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3566ae62-9dda-49e1-8e41-68502db0bcd9 None None] MFA Rules not processed for user `fb770d4379e74bf79ad4280329c771c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:30.033976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3566ae62-9dda-49e1-8e41-68502db0bcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:30.113289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:30.157355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 693/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:43:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:31.539093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62098-9124-40ab-9fbc-94f3847d2dd0 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:31.539712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62098-9124-40ab-9fbc-94f3847d2dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:31.539712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62098-9124-40ab-9fbc-94f3847d2dd0 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:31.541265 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 661/1354] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:31.546695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f795f-f4dc-4680-ba31-68fe341f5913 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:31.547172 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f795f-f4dc-4680-ba31-68fe341f5913 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:31.547295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f795f-f4dc-4680-ba31-68fe341f5913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:31.628929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-441f795f-f4dc-4680-ba31-68fe341f5913 None None] MFA Rules not processed for user `fb770d4379e74bf79ad4280329c771c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:31.630108 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-441f795f-f4dc-4680-ba31-68fe341f5913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:31.739762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 694/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:43:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:35.150735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509ef55-4189-48d3-ae8e-703014f598e8 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:35.151297 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509ef55-4189-48d3-ae8e-703014f598e8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:35.151297 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509ef55-4189-48d3-ae8e-703014f598e8 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:35.152601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 662/1356] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:35.157867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d82e9-636b-408d-a919-11c4f01525f0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:35.158325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d82e9-636b-408d-a919-11c4f01525f0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:35.158477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d82e9-636b-408d-a919-11c4f01525f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:35.196483 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:35.231614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4d3d82e9-636b-408d-a919-11c4f01525f0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:35.232849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4d3d82e9-636b-408d-a919-11c4f01525f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:35.374527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 695/1357] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:35.712694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55261332-5b62-43c3-8b53-67c414b0deda None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:35.713128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55261332-5b62-43c3-8b53-67c414b0deda None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:35.713611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55261332-5b62-43c3-8b53-67c414b0deda None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:35.715427 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 663/1358] 10.222.0.58 () {54 vars in 962 bytes} [Mon Aug 29 23:43:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:35.727504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d0849-eec6-4427-a66a-36908abcf2df None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:35.728419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d0849-eec6-4427-a66a-36908abcf2df None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:35.729246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d0849-eec6-4427-a66a-36908abcf2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:35.873755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8c2d0849-eec6-4427-a66a-36908abcf2df None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:35.875405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8c2d0849-eec6-4427-a66a-36908abcf2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:35.945912 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:36.039524 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 696/1359] 10.222.0.58 () {58 vars in 1059 bytes} [Mon Aug 29 23:43:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:36.048525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55261332-5b62-43c3-8b53-67c414b0deda None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:36.052644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-55261332-5b62-43c3-8b53-67c414b0deda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:36.358234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55261332-5b62-43c3-8b53-67c414b0deda None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:36.485009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:36.545993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d912b-b6d8-4098-9d99-8881cc0c1c2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:36.547068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d912b-b6d8-4098-9d99-8881cc0c1c2f service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:36.547397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d912b-b6d8-4098-9d99-8881cc0c1c2f service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:36.547617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d912b-b6d8-4098-9d99-8881cc0c1c2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:36.549385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a18d912b-b6d8-4098-9d99-8881cc0c1c2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:36.669184 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d912b-b6d8-4098-9d99-8881cc0c1c2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:36.671478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d912b-b6d8-4098-9d99-8881cc0c1c2f service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:36.700862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 664/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:43:39.344064 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28996b-a027-45cf-bebe-0ad14b36ec0e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:39.344829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28996b-a027-45cf-bebe-0ad14b36ec0e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:39.344829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28996b-a027-45cf-bebe-0ad14b36ec0e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:39.351336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 697/1361] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:43:39.353925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a09ae-f145-4a71-9f2f-35519aaba5f5 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:39.354663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a09ae-f145-4a71-9f2f-35519aaba5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:39.354663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a09ae-f145-4a71-9f2f-35519aaba5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:39.452442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-329a09ae-f145-4a71-9f2f-35519aaba5f5 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:39.454470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-329a09ae-f145-4a71-9f2f-35519aaba5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:39.524914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:39.622764 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 665/1362] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:40.163924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28996b-a027-45cf-bebe-0ad14b36ec0e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:40.166754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8e28996b-a027-45cf-bebe-0ad14b36ec0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:40.432776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28996b-a027-45cf-bebe-0ad14b36ec0e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:40.450919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:40.640397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c5aff-2cf4-455f-b44f-01bb28e0f027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:40.641936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c5aff-2cf4-455f-b44f-01bb28e0f027 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:40.642545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c5aff-2cf4-455f-b44f-01bb28e0f027 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:40.643092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c5aff-2cf4-455f-b44f-01bb28e0f027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:40.647184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d68c5aff-2cf4-455f-b44f-01bb28e0f027 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:40.784441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c5aff-2cf4-455f-b44f-01bb28e0f027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:40.787362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c5aff-2cf4-455f-b44f-01bb28e0f027 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:40.825392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 698/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:43:43.832855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d265b-b98d-48f9-a391-b1f766e03c47 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:43.833402 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d265b-b98d-48f9-a391-b1f766e03c47 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:43.833742 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d265b-b98d-48f9-a391-b1f766e03c47 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:43.835882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 666/1364] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:43.843596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c685a-7eb8-4ade-af4e-89af7669afc6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:43.844130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c685a-7eb8-4ade-af4e-89af7669afc6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:43.844540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c685a-7eb8-4ade-af4e-89af7669afc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:43.875862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:43.959465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-754c685a-7eb8-4ade-af4e-89af7669afc6 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:43.961052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-754c685a-7eb8-4ade-af4e-89af7669afc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:44.125458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 699/1365] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:44.448636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e6d8-edcf-4b83-ab73-bcbb40b92e05 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:44.449127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e6d8-edcf-4b83-ab73-bcbb40b92e05 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:44.449959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e6d8-edcf-4b83-ab73-bcbb40b92e05 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:44.452414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 667/1366] 10.222.0.58 () {54 vars in 962 bytes} [Mon Aug 29 23:43:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:44.461996 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd349bf1-a6bb-4cca-8921-4779bd288d68 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:44.462272 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd349bf1-a6bb-4cca-8921-4779bd288d68 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:44.462589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd349bf1-a6bb-4cca-8921-4779bd288d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:44.573418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-dd349bf1-a6bb-4cca-8921-4779bd288d68 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:44.575153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-dd349bf1-a6bb-4cca-8921-4779bd288d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:44.626821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:44.804707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 700/1367] 10.222.0.58 () {58 vars in 1059 bytes} [Mon Aug 29 23:43:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:44.817662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77e6d8-edcf-4b83-ab73-bcbb40b92e05 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:44.820460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7c77e6d8-edcf-4b83-ab73-bcbb40b92e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:45.007254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cb9a9-f2ff-4af1-aa13-4cc715f1dad9 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:45.007934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cb9a9-f2ff-4af1-aa13-4cc715f1dad9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:45.007934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cb9a9-f2ff-4af1-aa13-4cc715f1dad9 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:45.009049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77e6d8-edcf-4b83-ab73-bcbb40b92e05 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:45.011200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 701/1368] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:43:45.014209 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824e3a8-8fda-4665-9312-0689ad27958b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:45.014568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824e3a8-8fda-4665-9312-0689ad27958b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:45.014804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824e3a8-8fda-4665-9312-0689ad27958b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:45.033270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:45.100909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5824e3a8-8fda-4665-9312-0689ad27958b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:45.102128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5824e3a8-8fda-4665-9312-0689ad27958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:45.188110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b3a1e-01b6-4921-8cc5-60a8ac9fa798 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:45.189152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b3a1e-01b6-4921-8cc5-60a8ac9fa798 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:45.189654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b3a1e-01b6-4921-8cc5-60a8ac9fa798 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:45.190147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b3a1e-01b6-4921-8cc5-60a8ac9fa798 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:45.193266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-503b3a1e-01b6-4921-8cc5-60a8ac9fa798 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:45.218530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:45.229667 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 702/1369] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:43:45.343234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b3a1e-01b6-4921-8cc5-60a8ac9fa798 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:45.347738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b3a1e-01b6-4921-8cc5-60a8ac9fa798 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:45.381527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 668/1370] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:43:45.938465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e6ff2-be5a-4c07-b367-fb4aea27d4f3 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:45.938894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e6ff2-be5a-4c07-b367-fb4aea27d4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:45.939090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e6ff2-be5a-4c07-b367-fb4aea27d4f3 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:45.940700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 703/1371] 10.222.0.58 () {54 vars in 964 bytes} [Mon Aug 29 23:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:43:45.946260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa86c6e-46f0-4b3c-912f-b187c893813e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:45.946589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa86c6e-46f0-4b3c-912f-b187c893813e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:45.947051 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa86c6e-46f0-4b3c-912f-b187c893813e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:45.989782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:46.085111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-caa86c6e-46f0-4b3c-912f-b187c893813e None None] MFA Rules not processed for user `64a42591a6814a0cbcb8444829cb25b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:46.087406 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-caa86c6e-46f0-4b3c-912f-b187c893813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:46.214447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 669/1372] 10.222.0.58 () {58 vars in 1061 bytes} [Mon Aug 29 23:43:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:46.224113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e6ff2-be5a-4c07-b367-fb4aea27d4f3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:46.226013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ee3e6ff2-be5a-4c07-b367-fb4aea27d4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:46.406122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e6ff2-be5a-4c07-b367-fb4aea27d4f3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:46.528210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973422bb-9a74-41fd-824e-eaa781ff780b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:46.528679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973422bb-9a74-41fd-824e-eaa781ff780b service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:46.528944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973422bb-9a74-41fd-824e-eaa781ff780b service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:46.529203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973422bb-9a74-41fd-824e-eaa781ff780b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:46.530919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-973422bb-9a74-41fd-824e-eaa781ff780b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:46.600291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:46.685883 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973422bb-9a74-41fd-824e-eaa781ff780b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:46.687406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973422bb-9a74-41fd-824e-eaa781ff780b service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:46.707877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 704/1373] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:43:47.883091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db5c5-b7fa-465c-a7a5-1f1568397ad0 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:47.883668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db5c5-b7fa-465c-a7a5-1f1568397ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:47.883668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db5c5-b7fa-465c-a7a5-1f1568397ad0 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:47.884973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 670/1374] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:47.891205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87591a6-8b95-4fc2-8d80-2c6d1126dc17 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:47.891652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87591a6-8b95-4fc2-8d80-2c6d1126dc17 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:47.891825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87591a6-8b95-4fc2-8d80-2c6d1126dc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:47.960092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f87591a6-8b95-4fc2-8d80-2c6d1126dc17 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:47.961327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f87591a6-8b95-4fc2-8d80-2c6d1126dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:48.059619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 705/1375] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:48.354624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837db5c5-b7fa-465c-a7a5-1f1568397ad0 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:48.357221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-837db5c5-b7fa-465c-a7a5-1f1568397ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:48.513832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:48.558165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837db5c5-b7fa-465c-a7a5-1f1568397ad0 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:48.688125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00306d7-31a9-49f4-a139-d7d9fd74b16d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:48.688736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00306d7-31a9-49f4-a139-d7d9fd74b16d service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:48.689011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00306d7-31a9-49f4-a139-d7d9fd74b16d service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:48.689273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00306d7-31a9-49f4-a139-d7d9fd74b16d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:48.691558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c00306d7-31a9-49f4-a139-d7d9fd74b16d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:48.790813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:48.808965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00306d7-31a9-49f4-a139-d7d9fd74b16d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:48.808965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00306d7-31a9-49f4-a139-d7d9fd74b16d service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:48.833840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 671/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:43:50.981187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a22672-24da-4088-a62e-cafb9e295f93 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:50.981730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a22672-24da-4088-a62e-cafb9e295f93 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:50.981730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a22672-24da-4088-a62e-cafb9e295f93 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:50.983164 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 706/1377] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:50.989138 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc22342-f6e7-498e-a6cb-a72cb01ee807 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:50.989704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc22342-f6e7-498e-a6cb-a72cb01ee807 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:50.989784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc22342-f6e7-498e-a6cb-a72cb01ee807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:51.066945 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-5cc22342-f6e7-498e-a6cb-a72cb01ee807 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:51.068153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5cc22342-f6e7-498e-a6cb-a72cb01ee807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:51.179986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 672/1378] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:51.471471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a22672-24da-4088-a62e-cafb9e295f93 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:51.473582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f5a22672-24da-4088-a62e-cafb9e295f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:51.477168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:51.634247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a22672-24da-4088-a62e-cafb9e295f93 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:51.717406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:51.763939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e261d4-cc53-4d67-b87c-47d8245de94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:51.764561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e261d4-cc53-4d67-b87c-47d8245de94b service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:51.764839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e261d4-cc53-4d67-b87c-47d8245de94b service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:51.765109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e261d4-cc53-4d67-b87c-47d8245de94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:51.766750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f5e261d4-cc53-4d67-b87c-47d8245de94b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:51.879660 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e261d4-cc53-4d67-b87c-47d8245de94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:51.881368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e261d4-cc53-4d67-b87c-47d8245de94b service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:51.905898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 707/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:43:54.313333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847205-3bba-4d92-8b9a-b4fc4cd5bd42 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:54.313951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847205-3bba-4d92-8b9a-b4fc4cd5bd42 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:54.313951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847205-3bba-4d92-8b9a-b4fc4cd5bd42 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:54.315243 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 673/1380] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:54.321076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e46161-c258-4ba2-aaa8-3b33a5b7d3d5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:54.321573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e46161-c258-4ba2-aaa8-3b33a5b7d3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:54.321688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e46161-c258-4ba2-aaa8-3b33a5b7d3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:54.397057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-14e46161-c258-4ba2-aaa8-3b33a5b7d3d5 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:54.398492 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-14e46161-c258-4ba2-aaa8-3b33a5b7d3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:54.440878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:54.501786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 708/1381] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:43:54.806659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09847205-3bba-4d92-8b9a-b4fc4cd5bd42 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:54.808374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-09847205-3bba-4d92-8b9a-b4fc4cd5bd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:54.884814 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:54.984029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09847205-3bba-4d92-8b9a-b4fc4cd5bd42 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:55.104296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3479b92-5404-4e09-b5a4-be64ad2b64b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:55.104905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3479b92-5404-4e09-b5a4-be64ad2b64b3 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:55.105189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3479b92-5404-4e09-b5a4-be64ad2b64b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:55.105461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3479b92-5404-4e09-b5a4-be64ad2b64b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:55.107140 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b3479b92-5404-4e09-b5a4-be64ad2b64b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:55.127108 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:55.226779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3479b92-5404-4e09-b5a4-be64ad2b64b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:55.229020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3479b92-5404-4e09-b5a4-be64ad2b64b3 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:55.252385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 674/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:43:57.718058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e843b7f-c821-485c-83df-17b72a9eeba0 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:57.718711 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e843b7f-c821-485c-83df-17b72a9eeba0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:57.718711 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e843b7f-c821-485c-83df-17b72a9eeba0 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:57.719966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 709/1383] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:43:57.726166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76546c2-7ab7-4678-b3e2-a14f30c53d9f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:57.726881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76546c2-7ab7-4678-b3e2-a14f30c53d9f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:57.726881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76546c2-7ab7-4678-b3e2-a14f30c53d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:57.798316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-c76546c2-7ab7-4678-b3e2-a14f30c53d9f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:43:57.799515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c76546c2-7ab7-4678-b3e2-a14f30c53d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:57.904660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:57.913813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 675/1384] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:43:58.215859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e843b7f-c821-485c-83df-17b72a9eeba0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:43:58.217990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5e843b7f-c821-485c-83df-17b72a9eeba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:58.402942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e843b7f-c821-485c-83df-17b72a9eeba0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:43:58.426994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:43:58.560458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0f29f-60b0-4b70-90ae-958126810558 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:43:58.561001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0f29f-60b0-4b70-90ae-958126810558 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:43:58.561282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0f29f-60b0-4b70-90ae-958126810558 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:43:58.561554 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0f29f-60b0-4b70-90ae-958126810558 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:43:58.563309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d2b0f29f-60b0-4b70-90ae-958126810558 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:43:58.671641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0f29f-60b0-4b70-90ae-958126810558 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:43:58.673763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0f29f-60b0-4b70-90ae-958126810558 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:43:58.701211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 710/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:43:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:44:00.890240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa3b5e-54f0-416e-b3e0-6b9542597e7e None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:00.890893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa3b5e-54f0-416e-b3e0-6b9542597e7e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:00.890893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa3b5e-54f0-416e-b3e0-6b9542597e7e None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:00.892411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 676/1386] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:00.898844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8f697-9b0e-492c-903d-7183a2e9851e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:00.899381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8f697-9b0e-492c-903d-7183a2e9851e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:00.899671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8f697-9b0e-492c-903d-7183a2e9851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:00.939552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:00.977805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-72f8f697-9b0e-492c-903d-7183a2e9851e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:00.979621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-72f8f697-9b0e-492c-903d-7183a2e9851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:01.111283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 711/1387] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:01.407859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa3b5e-54f0-416e-b3e0-6b9542597e7e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:01.409973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-49aa3b5e-54f0-416e-b3e0-6b9542597e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:01.569407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa3b5e-54f0-416e-b3e0-6b9542597e7e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:01.644318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:01.700081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc9ea6-b771-441e-87d2-d10e5e3b0d56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:01.700687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc9ea6-b771-441e-87d2-d10e5e3b0d56 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:01.700979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc9ea6-b771-441e-87d2-d10e5e3b0d56 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:01.701241 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc9ea6-b771-441e-87d2-d10e5e3b0d56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:01.702899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-16bc9ea6-b771-441e-87d2-d10e5e3b0d56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:01.825400 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc9ea6-b771-441e-87d2-d10e5e3b0d56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:01.827070 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc9ea6-b771-441e-87d2-d10e5e3b0d56 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:01.847465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 677/1388] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:02.654399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8aa01-d4ae-4679-bb30-ab90586b810c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:02.654973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8aa01-d4ae-4679-bb30-ab90586b810c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:02.654973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8aa01-d4ae-4679-bb30-ab90586b810c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:02.656389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 712/1389] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:02.663454 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b8546-e58e-4294-87b0-28afa5beb0a4 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:02.663975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b8546-e58e-4294-87b0-28afa5beb0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:02.664134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b8546-e58e-4294-87b0-28afa5beb0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:02.734938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-826b8546-e58e-4294-87b0-28afa5beb0a4 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:02.736196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-826b8546-e58e-4294-87b0-28afa5beb0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:02.777411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:02.854662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 678/1390] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:03.163899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b598569-ea2b-4980-ab71-90b0deea408f None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:03.164202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b598569-ea2b-4980-ab71-90b0deea408f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:03.164621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b598569-ea2b-4980-ab71-90b0deea408f None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:03.166219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 713/1391] 10.222.0.58 () {54 vars in 967 bytes} [Mon Aug 29 23:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:03.171715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563c0c2-279a-45ff-b123-f551431a3941 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:03.172028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563c0c2-279a-45ff-b123-f551431a3941 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:03.172302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563c0c2-279a-45ff-b123-f551431a3941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:03.265942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-b563c0c2-279a-45ff-b123-f551431a3941 None None] MFA Rules not processed for user `d7a59337d20e4678b8a16ac35bffbdb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:03.268114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b563c0c2-279a-45ff-b123-f551431a3941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:03.378121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 679/1392] 10.222.0.58 () {58 vars in 1064 bytes} [Mon Aug 29 23:44:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:03.385677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b598569-ea2b-4980-ab71-90b0deea408f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:03.387129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8b598569-ea2b-4980-ab71-90b0deea408f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:03.506134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b598569-ea2b-4980-ab71-90b0deea408f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:03.514884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:03.590225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f0a5a-72eb-4c01-8cbc-523ce7d5586d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a59337d20e4678b8a16ac35bffbdb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:03.590896 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f0a5a-72eb-4c01-8cbc-523ce7d5586d service glance] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:03.591176 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f0a5a-72eb-4c01-8cbc-523ce7d5586d service glance] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:03.591471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f0a5a-72eb-4c01-8cbc-523ce7d5586d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:03.593196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-af3f0a5a-72eb-4c01-8cbc-523ce7d5586d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:03.719815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f0a5a-72eb-4c01-8cbc-523ce7d5586d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:03.721198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f0a5a-72eb-4c01-8cbc-523ce7d5586d service glance] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:03.744694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 714/1393] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:04.004007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e57a3-2a69-4a77-9cd4-06a9187cc3ed None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:04.004571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e57a3-2a69-4a77-9cd4-06a9187cc3ed None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:04.004907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e57a3-2a69-4a77-9cd4-06a9187cc3ed None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:04.006335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 680/1394] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:44:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:04.012425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca389ea8-d23e-40ea-b2bf-f514b61df151 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:04.012712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca389ea8-d23e-40ea-b2bf-f514b61df151 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:04.013037 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca389ea8-d23e-40ea-b2bf-f514b61df151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:04.068949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:04.095005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ca389ea8-d23e-40ea-b2bf-f514b61df151 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:04.096263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ca389ea8-d23e-40ea-b2bf-f514b61df151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:04.224905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 715/1395] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:04.506693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e57a3-2a69-4a77-9cd4-06a9187cc3ed None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:04.508253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5e7e57a3-2a69-4a77-9cd4-06a9187cc3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:04.544658 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:04.689228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e57a3-2a69-4a77-9cd4-06a9187cc3ed None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:04.773932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5ab26-4526-4066-8081-b25e6d8f00dc None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:04.774514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5ab26-4526-4066-8081-b25e6d8f00dc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:04.774514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5ab26-4526-4066-8081-b25e6d8f00dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:04.803181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:04.815043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d284844-761f-4eda-901c-dbdcbb8eb9d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:04.816389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d284844-761f-4eda-901c-dbdcbb8eb9d4 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:04.816667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d284844-761f-4eda-901c-dbdcbb8eb9d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:04.817044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d284844-761f-4eda-901c-dbdcbb8eb9d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:04.818719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4d284844-761f-4eda-901c-dbdcbb8eb9d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:04.849375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-fde5ab26-4526-4066-8081-b25e6d8f00dc None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:04.850671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fde5ab26-4526-4066-8081-b25e6d8f00dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:04.916241 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d284844-761f-4eda-901c-dbdcbb8eb9d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:04.917793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d284844-761f-4eda-901c-dbdcbb8eb9d4 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:04.929817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:04.937850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 681/1396] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:04.956702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 716/1397] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:44:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:44:05.007943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf47a02-0d89-42cf-85ab-98c8b8c69da7 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:05.008508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf47a02-0d89-42cf-85ab-98c8b8c69da7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:05.008508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf47a02-0d89-42cf-85ab-98c8b8c69da7 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:05.009846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 682/1398] 10.222.0.58 () {54 vars in 943 bytes} [Mon Aug 29 23:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:05.017320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbc44f-b9f8-444e-b9bd-2f41e32d62d2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:05.017605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbc44f-b9f8-444e-b9bd-2f41e32d62d2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:05.017879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbc44f-b9f8-444e-b9bd-2f41e32d62d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:05.107511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-38bbc44f-b9f8-444e-b9bd-2f41e32d62d2 None None] MFA Rules not processed for user `c6b37b09a1ae45e8bc16adfa59500eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:05.108895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-38bbc44f-b9f8-444e-b9bd-2f41e32d62d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:05.242625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 717/1399] 10.222.0.58 () {58 vars in 1040 bytes} [Mon Aug 29 23:44:05 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:05.253723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf47a02-0d89-42cf-85ab-98c8b8c69da7 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:05.255558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-daf47a02-0d89-42cf-85ab-98c8b8c69da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:05.375706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:05.405267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf47a02-0d89-42cf-85ab-98c8b8c69da7 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:05.523889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263674c5-85e4-49ea-bc13-edda96ac315d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:05.524408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263674c5-85e4-49ea-bc13-edda96ac315d service swift] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:05.524687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263674c5-85e4-49ea-bc13-edda96ac315d service swift] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:05.524958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263674c5-85e4-49ea-bc13-edda96ac315d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:05.526654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-263674c5-85e4-49ea-bc13-edda96ac315d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:05.617476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263674c5-85e4-49ea-bc13-edda96ac315d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:05.619126 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263674c5-85e4-49ea-bc13-edda96ac315d service swift] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:05.627816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 683/1400] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:44:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:44:07.140099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc786829-d26a-4d3c-b2ca-a0122c9567cf None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:07.140099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc786829-d26a-4d3c-b2ca-a0122c9567cf None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:07.140099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc786829-d26a-4d3c-b2ca-a0122c9567cf None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:07.140099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 718/1401] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:07.140977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad321d08-0665-432e-a109-c6e8b6a00759 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:07.140977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad321d08-0665-432e-a109-c6e8b6a00759 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:07.140977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad321d08-0665-432e-a109-c6e8b6a00759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:07.157331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:07.218169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-ad321d08-0665-432e-a109-c6e8b6a00759 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:07.219587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ad321d08-0665-432e-a109-c6e8b6a00759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:07.384194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 684/1402] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:07.696772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc786829-d26a-4d3c-b2ca-a0122c9567cf None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:07.699110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cc786829-d26a-4d3c-b2ca-a0122c9567cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:07.757780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:07.953518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc786829-d26a-4d3c-b2ca-a0122c9567cf None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:08.151524 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4ca58-9bd5-4103-8eb7-744d969da0e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:08.152593 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4ca58-9bd5-4103-8eb7-744d969da0e0 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:08.152593 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4ca58-9bd5-4103-8eb7-744d969da0e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:08.152884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4ca58-9bd5-4103-8eb7-744d969da0e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:08.154359 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-07c4ca58-9bd5-4103-8eb7-744d969da0e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:08.260075 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4ca58-9bd5-4103-8eb7-744d969da0e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:08.261715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4ca58-9bd5-4103-8eb7-744d969da0e0 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:08.289660 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 719/1403] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:08.314314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:10.433888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2afda3-fff5-465f-8185-dbdd83eb61ae None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:10.434599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2afda3-fff5-465f-8185-dbdd83eb61ae None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:10.434599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2afda3-fff5-465f-8185-dbdd83eb61ae None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:10.435793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 685/1404] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:10.441629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b4178-d230-4eb9-ae17-86dc8b5c2a85 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:10.442265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b4178-d230-4eb9-ae17-86dc8b5c2a85 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:10.442265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b4178-d230-4eb9-ae17-86dc8b5c2a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:10.519210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-be7b4178-d230-4eb9-ae17-86dc8b5c2a85 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:10.520682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-be7b4178-d230-4eb9-ae17-86dc8b5c2a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:10.607979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:10.647171 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 720/1405] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:10.962649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2afda3-fff5-465f-8185-dbdd83eb61ae None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:10.964443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bc2afda3-fff5-465f-8185-dbdd83eb61ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:11.084339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:11.178610 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2afda3-fff5-465f-8185-dbdd83eb61ae None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:11.310703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7259cbf-0ebc-4757-ac19-3bfbef03bf09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:11.311260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7259cbf-0ebc-4757-ac19-3bfbef03bf09 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:11.311544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7259cbf-0ebc-4757-ac19-3bfbef03bf09 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:11.311820 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7259cbf-0ebc-4757-ac19-3bfbef03bf09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:11.313443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b7259cbf-0ebc-4757-ac19-3bfbef03bf09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:11.382958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:11.423176 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7259cbf-0ebc-4757-ac19-3bfbef03bf09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:11.425581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7259cbf-0ebc-4757-ac19-3bfbef03bf09 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:11.451841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 686/1406] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:13.692644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439d69d-d0cd-43d1-98d2-495d3538a467 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:13.692644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439d69d-d0cd-43d1-98d2-495d3538a467 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:13.692644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439d69d-d0cd-43d1-98d2-495d3538a467 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:13.693838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 721/1407] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:13.701990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83ad5f-6e7f-4ba8-a6cf-008af6911ce7 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:13.702913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83ad5f-6e7f-4ba8-a6cf-008af6911ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:13.703187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83ad5f-6e7f-4ba8-a6cf-008af6911ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:13.782016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-af83ad5f-6e7f-4ba8-a6cf-008af6911ce7 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:13.783338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-af83ad5f-6e7f-4ba8-a6cf-008af6911ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:13.911944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 687/1408] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:44:14.208630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439d69d-d0cd-43d1-98d2-495d3538a467 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:14.210166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8439d69d-d0cd-43d1-98d2-495d3538a467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:14.388327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439d69d-d0cd-43d1-98d2-495d3538a467 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:14.452745 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:14.520286 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38ca33-a3fb-45e0-b7c0-72e3e263f329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:14.520877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38ca33-a3fb-45e0-b7c0-72e3e263f329 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:14.521167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38ca33-a3fb-45e0-b7c0-72e3e263f329 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:14.521441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38ca33-a3fb-45e0-b7c0-72e3e263f329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:14.523124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-dc38ca33-a3fb-45e0-b7c0-72e3e263f329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:14.643463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38ca33-a3fb-45e0-b7c0-72e3e263f329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:14.645515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38ca33-a3fb-45e0-b7c0-72e3e263f329 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:14.676098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 722/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:23.909492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=87 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:24.666088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=72 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:30.149899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba7d3-ee88-41a0-87a6-74c317f81512 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:30.150512 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba7d3-ee88-41a0-87a6-74c317f81512 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:30.150512 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba7d3-ee88-41a0-87a6-74c317f81512 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:30.151864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 688/1410] 10.222.0.58 () {54 vars in 918 bytes} [Mon Aug 29 23:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:30.158629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c67b00-46fc-4071-a1c4-08dd291b6ae1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:30.159332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c67b00-46fc-4071-a1c4-08dd291b6ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:30.159591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c67b00-46fc-4071-a1c4-08dd291b6ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:30.246511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b9c67b00-46fc-4071-a1c4-08dd291b6ae1 None None] MFA Rules not processed for user `fb770d4379e74bf79ad4280329c771c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:30.247731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b9c67b00-46fc-4071-a1c4-08dd291b6ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:30.361631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 723/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Mon Aug 29 23:44:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:30.377749 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8401c-1533-400b-8b39-dbea4582d605 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:30.378123 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8401c-1533-400b-8b39-dbea4582d605 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:30.378407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8401c-1533-400b-8b39-dbea4582d605 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:30.379762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 689/1412] 10.222.0.58 () {54 vars in 918 bytes} [Mon Aug 29 23:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:30.384628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72318c0-d554-494c-8036-7d014a920e96 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:30.384906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72318c0-d554-494c-8036-7d014a920e96 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:30.385188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72318c0-d554-494c-8036-7d014a920e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:30.464580 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:30.467718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d72318c0-d554-494c-8036-7d014a920e96 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:30.468972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d72318c0-d554-494c-8036-7d014a920e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:30.592743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 724/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Mon Aug 29 23:44:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:33.125631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939eefc-71af-44fd-b3e0-f096314eed44 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:33.126172 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939eefc-71af-44fd-b3e0-f096314eed44 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:33.126172 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939eefc-71af-44fd-b3e0-f096314eed44 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:33.127674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 690/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:33.134001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6cc9e-946f-4ea3-ba35-05c3a45898ab None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:33.134451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6cc9e-946f-4ea3-ba35-05c3a45898ab None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:33.134579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6cc9e-946f-4ea3-ba35-05c3a45898ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:33.223343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-88e6cc9e-946f-4ea3-ba35-05c3a45898ab None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:33.224569 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-88e6cc9e-946f-4ea3-ba35-05c3a45898ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:33.282179 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:33.327523 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 725/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:33.623805 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939eefc-71af-44fd-b3e0-f096314eed44 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:33.625311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d939eefc-71af-44fd-b3e0-f096314eed44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:33.779369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939eefc-71af-44fd-b3e0-f096314eed44 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:33.871818 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:33.930151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cc676-108b-4744-a533-20555fe383f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:33.930797 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cc676-108b-4744-a533-20555fe383f7 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:33.931075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cc676-108b-4744-a533-20555fe383f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:33.931345 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cc676-108b-4744-a533-20555fe383f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:33.932986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-0d1cc676-108b-4744-a533-20555fe383f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:34.031229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cc676-108b-4744-a533-20555fe383f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:34.032738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cc676-108b-4744-a533-20555fe383f7 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:34.053518 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 691/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:34.984331 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbe150-2a0a-42f9-b477-2a482bd8f6ff None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:34.984863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbe150-2a0a-42f9-b477-2a482bd8f6ff None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:34.984863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbe150-2a0a-42f9-b477-2a482bd8f6ff None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:34.986157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 726/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:34.991319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93850211-1ddc-48c5-8cc9-b617831ca88c None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:34.991754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93850211-1ddc-48c5-8cc9-b617831ca88c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:34.991847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93850211-1ddc-48c5-8cc9-b617831ca88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:35.073437 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-93850211-1ddc-48c5-8cc9-b617831ca88c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:35.074715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-93850211-1ddc-48c5-8cc9-b617831ca88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:35.103677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:35.177911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 692/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:35.338848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbe150-2a0a-42f9-b477-2a482bd8f6ff None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:35.340305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-49bbe150-2a0a-42f9-b477-2a482bd8f6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:35.494774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbe150-2a0a-42f9-b477-2a482bd8f6ff None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:35.541931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:35.618392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b0b81-21a2-4948-a488-c4e2badd0426 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:35.619033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b0b81-21a2-4948-a488-c4e2badd0426 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:35.619307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b0b81-21a2-4948-a488-c4e2badd0426 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:35.619572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b0b81-21a2-4948-a488-c4e2badd0426 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:35.621176 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9e5b0b81-21a2-4948-a488-c4e2badd0426 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:35.718742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b0b81-21a2-4948-a488-c4e2badd0426 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:35.720376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b0b81-21a2-4948-a488-c4e2badd0426 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:35.741632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 727/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:36.691480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54c613-0c5e-4db9-a06b-78f4f576ddf5 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:36.692056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54c613-0c5e-4db9-a06b-78f4f576ddf5 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:36.692056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54c613-0c5e-4db9-a06b-78f4f576ddf5 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:36.693378 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 693/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:44:36.699261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff8475-5c8b-448f-9cc8-4b467d508bee None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:36.700068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff8475-5c8b-448f-9cc8-4b467d508bee None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:36.700498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff8475-5c8b-448f-9cc8-4b467d508bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:36.764424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:36.790237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9aff8475-5c8b-448f-9cc8-4b467d508bee None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:36.791515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9aff8475-5c8b-448f-9cc8-4b467d508bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:36.907681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 728/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:44:37.065463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54c613-0c5e-4db9-a06b-78f4f576ddf5 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:37.067339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ef54c613-0c5e-4db9-a06b-78f4f576ddf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:37.271431 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54c613-0c5e-4db9-a06b-78f4f576ddf5 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:37.285480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:37.402202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6509f0-2f4e-497b-9f2a-37e00be2b2bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:37.402859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6509f0-2f4e-497b-9f2a-37e00be2b2bf service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:37.403143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6509f0-2f4e-497b-9f2a-37e00be2b2bf service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:37.403418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6509f0-2f4e-497b-9f2a-37e00be2b2bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:37.405104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-fc6509f0-2f4e-497b-9f2a-37e00be2b2bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:37.507617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6509f0-2f4e-497b-9f2a-37e00be2b2bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:37.509303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6509f0-2f4e-497b-9f2a-37e00be2b2bf service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:37.530101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 694/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:38.470703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e15e71-6b18-421b-86d7-7ec2ac129a61 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:38.478017 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e15e71-6b18-421b-86d7-7ec2ac129a61 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:38.478017 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e15e71-6b18-421b-86d7-7ec2ac129a61 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:38.478017 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 729/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:38.479272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17fa0f-2873-4a43-99ec-2b6f1ea974b4 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:38.480033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17fa0f-2873-4a43-99ec-2b6f1ea974b4 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:38.480033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17fa0f-2873-4a43-99ec-2b6f1ea974b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:38.502214 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:38.568594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-8c17fa0f-2873-4a43-99ec-2b6f1ea974b4 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:38.569842 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8c17fa0f-2873-4a43-99ec-2b6f1ea974b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:38.704063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 695/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:44:38.867227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e15e71-6b18-421b-86d7-7ec2ac129a61 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:38.868997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-94e15e71-6b18-421b-86d7-7ec2ac129a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:38.993983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:39.038568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e15e71-6b18-421b-86d7-7ec2ac129a61 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:39.233652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ecd6ce-9376-4ee5-be8a-7e5d8bdad6a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:39.234280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecd6ce-9376-4ee5-be8a-7e5d8bdad6a5 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:39.234595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecd6ce-9376-4ee5-be8a-7e5d8bdad6a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:39.234866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecd6ce-9376-4ee5-be8a-7e5d8bdad6a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:39.236585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-94ecd6ce-9376-4ee5-be8a-7e5d8bdad6a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:39.339796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:39.356376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ecd6ce-9376-4ee5-be8a-7e5d8bdad6a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:39.358185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ecd6ce-9376-4ee5-be8a-7e5d8bdad6a5 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:39.381247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 730/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:44:40.333219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f417f2-2e65-410d-8d7a-d0dd6518df6e None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:40.333784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f417f2-2e65-410d-8d7a-d0dd6518df6e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:40.333784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f417f2-2e65-410d-8d7a-d0dd6518df6e None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:40.335109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 696/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:40.340843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5ab09-9269-4a18-88c9-991a03d35a70 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:40.341399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5ab09-9269-4a18-88c9-991a03d35a70 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:40.341399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5ab09-9269-4a18-88c9-991a03d35a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:40.426341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-99e5ab09-9269-4a18-88c9-991a03d35a70 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:40.427633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-99e5ab09-9269-4a18-88c9-991a03d35a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:40.549927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 731/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:40.705385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f417f2-2e65-410d-8d7a-d0dd6518df6e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:40.708064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-79f417f2-2e65-410d-8d7a-d0dd6518df6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:40.799309 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:40.866258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f417f2-2e65-410d-8d7a-d0dd6518df6e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:40.999833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496b841-62a2-4d8b-bad3-33b8e89799fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:40.999833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496b841-62a2-4d8b-bad3-33b8e89799fc service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:40.999833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496b841-62a2-4d8b-bad3-33b8e89799fc service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:40.999833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496b841-62a2-4d8b-bad3-33b8e89799fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:41.000477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2496b841-62a2-4d8b-bad3-33b8e89799fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:41.062596 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:41.111638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496b841-62a2-4d8b-bad3-33b8e89799fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:41.113617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496b841-62a2-4d8b-bad3-33b8e89799fc service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:41.148931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 697/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:42.194594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1d8a3-600d-453b-90d7-97b44ef02bde None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:42.195417 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1d8a3-600d-453b-90d7-97b44ef02bde None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:42.195619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1d8a3-600d-453b-90d7-97b44ef02bde None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:42.197191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 732/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:44:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:42.203208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b367c-c7bd-4051-aab0-a3e74f5514fb None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:42.203751 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b367c-c7bd-4051-aab0-a3e74f5514fb None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:42.204184 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b367c-c7bd-4051-aab0-a3e74f5514fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:42.297433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d42b367c-c7bd-4051-aab0-a3e74f5514fb None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:42.298700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d42b367c-c7bd-4051-aab0-a3e74f5514fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:42.406681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 698/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:42.567534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1d8a3-600d-453b-90d7-97b44ef02bde None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:42.568981 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-eeb1d8a3-600d-453b-90d7-97b44ef02bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:42.624019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:42.748168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1d8a3-600d-453b-90d7-97b44ef02bde None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:42.876060 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6308907-f0fb-4ba0-aa8f-0d5e34e787d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:42.876671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6308907-f0fb-4ba0-aa8f-0d5e34e787d0 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:42.876945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6308907-f0fb-4ba0-aa8f-0d5e34e787d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:42.877214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6308907-f0fb-4ba0-aa8f-0d5e34e787d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:42.879519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d6308907-f0fb-4ba0-aa8f-0d5e34e787d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:42.887295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:42.981453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6308907-f0fb-4ba0-aa8f-0d5e34e787d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:42.983118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6308907-f0fb-4ba0-aa8f-0d5e34e787d0 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:43.003906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 733/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:43.928402 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27955f80-0852-4511-972c-5477c288e029 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:43.928960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27955f80-0852-4511-972c-5477c288e029 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:43.928960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27955f80-0852-4511-972c-5477c288e029 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:43.930277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 699/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:43.935349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce475-7445-459a-96c2-38ef9859dacb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:43.935803 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce475-7445-459a-96c2-38ef9859dacb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:43.935898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce475-7445-459a-96c2-38ef9859dacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:44.015837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-168ce475-7445-459a-96c2-38ef9859dacb None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:44.017059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-168ce475-7445-459a-96c2-38ef9859dacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:44.100438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:44.135443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 734/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:44.289275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27955f80-0852-4511-972c-5477c288e029 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:44.290889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-27955f80-0852-4511-972c-5477c288e029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:44.309614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:44.504101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27955f80-0852-4511-972c-5477c288e029 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:44.556918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:44.584660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58c91b-a879-4cf8-9c51-d7d6c8ec4206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:44.585261 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58c91b-a879-4cf8-9c51-d7d6c8ec4206 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:44.585572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58c91b-a879-4cf8-9c51-d7d6c8ec4206 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:44.585804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58c91b-a879-4cf8-9c51-d7d6c8ec4206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:44.587448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-cc58c91b-a879-4cf8-9c51-d7d6c8ec4206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:44.682561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58c91b-a879-4cf8-9c51-d7d6c8ec4206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:44.684168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58c91b-a879-4cf8-9c51-d7d6c8ec4206 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:44.703023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 700/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:45.609649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8c905-9f11-4179-aa40-1cc6821612ec None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:45.610192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8c905-9f11-4179-aa40-1cc6821612ec None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:45.610192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8c905-9f11-4179-aa40-1cc6821612ec None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:45.611523 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 735/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:45.617633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134fa4-b1ae-4107-97d4-e557aab95058 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:45.618139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134fa4-b1ae-4107-97d4-e557aab95058 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:45.618259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134fa4-b1ae-4107-97d4-e557aab95058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:45.705527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-58134fa4-b1ae-4107-97d4-e557aab95058 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:45.706897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-58134fa4-b1ae-4107-97d4-e557aab95058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:45.749612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:45.811766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 701/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:45.969787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8c905-9f11-4179-aa40-1cc6821612ec None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:45.971556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-80c8c905-9f11-4179-aa40-1cc6821612ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:46.132087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8c905-9f11-4179-aa40-1cc6821612ec None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:46.191048 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:46.260648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef9a28-bbd6-4f86-af91-d9478f1da72d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:46.261261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9a28-bbd6-4f86-af91-d9478f1da72d service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:46.261537 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9a28-bbd6-4f86-af91-d9478f1da72d service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:46.261807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9a28-bbd6-4f86-af91-d9478f1da72d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:46.263478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-86ef9a28-bbd6-4f86-af91-d9478f1da72d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:46.365913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef9a28-bbd6-4f86-af91-d9478f1da72d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:46.367773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef9a28-bbd6-4f86-af91-d9478f1da72d service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:46.403806 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 736/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:47.368364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cd6b9-98a5-4010-8769-1978ccd1b129 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:47.369166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cd6b9-98a5-4010-8769-1978ccd1b129 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:47.369166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cd6b9-98a5-4010-8769-1978ccd1b129 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:47.370256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 702/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:47.375534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445f1a7-0200-4bc1-bf94-6002a640a45b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:47.376022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445f1a7-0200-4bc1-bf94-6002a640a45b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:47.376171 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445f1a7-0200-4bc1-bf94-6002a640a45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:47.457568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e445f1a7-0200-4bc1-bf94-6002a640a45b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:47.458729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e445f1a7-0200-4bc1-bf94-6002a640a45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:47.460974 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:47.565742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 737/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:47.726136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cd6b9-98a5-4010-8769-1978ccd1b129 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:47.727653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3d8cd6b9-98a5-4010-8769-1978ccd1b129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:47.885230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cd6b9-98a5-4010-8769-1978ccd1b129 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:47.914491 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:48.030591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c8ad5-e53e-4783-8f84-bae6d87581ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:48.031122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c8ad5-e53e-4783-8f84-bae6d87581ad service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:48.031406 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c8ad5-e53e-4783-8f84-bae6d87581ad service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:48.031680 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c8ad5-e53e-4783-8f84-bae6d87581ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:48.033330 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1d8c8ad5-e53e-4783-8f84-bae6d87581ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:48.134144 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c8ad5-e53e-4783-8f84-bae6d87581ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:48.135876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c8ad5-e53e-4783-8f84-bae6d87581ad service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:48.156624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 703/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:50.059942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99a4e-7432-4cd1-939f-d2f60c6e7678 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:50.060498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99a4e-7432-4cd1-939f-d2f60c6e7678 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:50.060498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99a4e-7432-4cd1-939f-d2f60c6e7678 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:50.061833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 738/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:50.067347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e3ee5-eb00-419a-a948-a654e253614b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:50.067872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e3ee5-eb00-419a-a948-a654e253614b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:50.067872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e3ee5-eb00-419a-a948-a654e253614b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:50.113295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:50.140524 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-de8e3ee5-eb00-419a-a948-a654e253614b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:50.142251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-de8e3ee5-eb00-419a-a948-a654e253614b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:50.249720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 704/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:50.541625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a99a4e-7432-4cd1-939f-d2f60c6e7678 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:50.543234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a6a99a4e-7432-4cd1-939f-d2f60c6e7678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:50.667778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a99a4e-7432-4cd1-939f-d2f60c6e7678 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:50.702383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:50.747061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2c742-00ee-48c4-b61b-7c368c2e5237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:50.747676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2c742-00ee-48c4-b61b-7c368c2e5237 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:50.747950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2c742-00ee-48c4-b61b-7c368c2e5237 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:50.748217 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2c742-00ee-48c4-b61b-7c368c2e5237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:50.749832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-77b2c742-00ee-48c4-b61b-7c368c2e5237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:50.864732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2c742-00ee-48c4-b61b-7c368c2e5237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:50.866179 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2c742-00ee-48c4-b61b-7c368c2e5237 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:50.885425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 739/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:53.044166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723659-e367-4b5e-aa4f-16b0af7e0646 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:53.044762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723659-e367-4b5e-aa4f-16b0af7e0646 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:53.044762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723659-e367-4b5e-aa4f-16b0af7e0646 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:53.046094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 705/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:53.052234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c248dbd-9639-4e65-a4ea-6bdcd0992f2d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:53.052720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c248dbd-9639-4e65-a4ea-6bdcd0992f2d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:53.052829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c248dbd-9639-4e65-a4ea-6bdcd0992f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:53.121148 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5c248dbd-9639-4e65-a4ea-6bdcd0992f2d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:53.122379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5c248dbd-9639-4e65-a4ea-6bdcd0992f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:53.136282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:53.223360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 740/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:53.510926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb723659-e367-4b5e-aa4f-16b0af7e0646 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:53.512451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-fb723659-e367-4b5e-aa4f-16b0af7e0646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:53.623494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb723659-e367-4b5e-aa4f-16b0af7e0646 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:53.635894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:53.699678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67656ee8-f41c-41e3-b231-ee88e21abbd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:53.700279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656ee8-f41c-41e3-b231-ee88e21abbd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:53.700557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656ee8-f41c-41e3-b231-ee88e21abbd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:53.701091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656ee8-f41c-41e3-b231-ee88e21abbd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:53.702713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-67656ee8-f41c-41e3-b231-ee88e21abbd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:53.816231 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67656ee8-f41c-41e3-b231-ee88e21abbd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:53.817740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67656ee8-f41c-41e3-b231-ee88e21abbd7 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:53.838540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 706/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:56.310786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5840e-fd91-4871-bbca-207b1a587548 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:56.311572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5840e-fd91-4871-bbca-207b1a587548 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:56.311572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5840e-fd91-4871-bbca-207b1a587548 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:56.313339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 741/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:56.319374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d76280-871b-4521-b213-944225eb8412 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:56.320195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d76280-871b-4521-b213-944225eb8412 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:56.320195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d76280-871b-4521-b213-944225eb8412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:56.379489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:56.400975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-02d76280-871b-4521-b213-944225eb8412 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:56.402236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-02d76280-871b-4521-b213-944225eb8412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:56.510187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 707/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:56.815822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5840e-fd91-4871-bbca-207b1a587548 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:56.817446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e7c5840e-fd91-4871-bbca-207b1a587548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:56.929418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5840e-fd91-4871-bbca-207b1a587548 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:56.972132 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:57.011364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4701b-62fb-4120-96b7-31479d5241d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:44:57.011973 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4701b-62fb-4120-96b7-31479d5241d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:57.012248 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4701b-62fb-4120-96b7-31479d5241d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:57.012516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4701b-62fb-4120-96b7-31479d5241d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:57.014126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d5a4701b-62fb-4120-96b7-31479d5241d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:57.149547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4701b-62fb-4120-96b7-31479d5241d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:44:57.151050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4701b-62fb-4120-96b7-31479d5241d1 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:44:57.171251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 742/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:44:59.332305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce27017-d86e-4e42-bbf4-6b60a691527b None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:59.332870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce27017-d86e-4e42-bbf4-6b60a691527b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:59.332870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce27017-d86e-4e42-bbf4-6b60a691527b None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:59.334194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 708/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:44:59.339835 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8c79b-f26a-481d-83e1-ba1d5a141657 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:44:59.340375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8c79b-f26a-481d-83e1-ba1d5a141657 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:44:59.340375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8c79b-f26a-481d-83e1-ba1d5a141657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:44:59.421300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ccf8c79b-f26a-481d-83e1-ba1d5a141657 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:44:59.422671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ccf8c79b-f26a-481d-83e1-ba1d5a141657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:59.447960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:44:59.551263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 743/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:44:59.844936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce27017-d86e-4e42-bbf4-6b60a691527b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:44:59.847028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9ce27017-d86e-4e42-bbf4-6b60a691527b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:44:59.964771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce27017-d86e-4e42-bbf4-6b60a691527b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:44:59.989545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:00.051943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4e430-7466-4c9c-a0d8-8dc90e207547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:00.052563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4e430-7466-4c9c-a0d8-8dc90e207547 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:00.052847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4e430-7466-4c9c-a0d8-8dc90e207547 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:00.053123 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4e430-7466-4c9c-a0d8-8dc90e207547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:00.054895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-61d4e430-7466-4c9c-a0d8-8dc90e207547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:00.177109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4e430-7466-4c9c-a0d8-8dc90e207547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:00.178776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4e430-7466-4c9c-a0d8-8dc90e207547 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:00.198863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 709/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:02.324184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25182194-a59c-4825-932b-5e200f3b0013 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:02.324726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25182194-a59c-4825-932b-5e200f3b0013 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:02.324726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25182194-a59c-4825-932b-5e200f3b0013 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:02.325979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 744/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:02.332401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999636df-8a7f-45a2-ab0d-25fc19a94ac8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:02.332893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999636df-8a7f-45a2-ab0d-25fc19a94ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:02.332986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999636df-8a7f-45a2-ab0d-25fc19a94ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:02.435314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:02.435314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-999636df-8a7f-45a2-ab0d-25fc19a94ac8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:02.435314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-999636df-8a7f-45a2-ab0d-25fc19a94ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:02.522037 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 710/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:02.819564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25182194-a59c-4825-932b-5e200f3b0013 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:02.821246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-25182194-a59c-4825-932b-5e200f3b0013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:02.934278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25182194-a59c-4825-932b-5e200f3b0013 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:02.998282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:03.026478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195aa14a-0d1b-4ece-85cc-1a64e9803312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:03.027087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195aa14a-0d1b-4ece-85cc-1a64e9803312 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:03.027368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195aa14a-0d1b-4ece-85cc-1a64e9803312 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:03.027629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195aa14a-0d1b-4ece-85cc-1a64e9803312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:03.029263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-195aa14a-0d1b-4ece-85cc-1a64e9803312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:03.161809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195aa14a-0d1b-4ece-85cc-1a64e9803312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:03.163331 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195aa14a-0d1b-4ece-85cc-1a64e9803312 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:03.183596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 745/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:05.318367 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98cc37-d2d6-4dd4-b908-eb603fba7bf4 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:05.319014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98cc37-d2d6-4dd4-b908-eb603fba7bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:05.319014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98cc37-d2d6-4dd4-b908-eb603fba7bf4 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:05.320376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 711/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:05.327050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b40b4-a96e-4c71-a78f-3fa67e917a06 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:05.327633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b40b4-a96e-4c71-a78f-3fa67e917a06 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:05.327633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b40b4-a96e-4c71-a78f-3fa67e917a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:05.404348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a70b40b4-a96e-4c71-a78f-3fa67e917a06 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:05.405565 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a70b40b4-a96e-4c71-a78f-3fa67e917a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:05.436683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:05.514496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 746/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:05.812187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98cc37-d2d6-4dd4-b908-eb603fba7bf4 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:05.814037 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-fc98cc37-d2d6-4dd4-b908-eb603fba7bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:05.952185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98cc37-d2d6-4dd4-b908-eb603fba7bf4 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:05.984575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:06.074302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3939c-2c94-473f-a793-6ff341b578e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:06.075770 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3939c-2c94-473f-a793-6ff341b578e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:06.076558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3939c-2c94-473f-a793-6ff341b578e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:06.077043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3939c-2c94-473f-a793-6ff341b578e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:06.081244 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-94b3939c-2c94-473f-a793-6ff341b578e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:06.214557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3939c-2c94-473f-a793-6ff341b578e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:06.216012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3939c-2c94-473f-a793-6ff341b578e0 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:06.235806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 712/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:08.280292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e498cf-dd28-4955-8012-3249f914a987 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:08.280858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e498cf-dd28-4955-8012-3249f914a987 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:08.281112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e498cf-dd28-4955-8012-3249f914a987 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:08.283284 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 747/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:45:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:08.291205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de2f95-e86c-415d-bb70-dffc5fec7c7e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:08.292029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de2f95-e86c-415d-bb70-dffc5fec7c7e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:08.292179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de2f95-e86c-415d-bb70-dffc5fec7c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:08.376192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-82de2f95-e86c-415d-bb70-dffc5fec7c7e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:08.378125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-82de2f95-e86c-415d-bb70-dffc5fec7c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:08.387836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:08.500609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 713/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:08.796813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e498cf-dd28-4955-8012-3249f914a987 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:08.798341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-28e498cf-dd28-4955-8012-3249f914a987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:08.910687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e498cf-dd28-4955-8012-3249f914a987 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:08.974111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:08.994805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ff316-eab3-4d3c-9b71-8b6ce51fd5d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:08.995412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ff316-eab3-4d3c-9b71-8b6ce51fd5d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:08.995688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ff316-eab3-4d3c-9b71-8b6ce51fd5d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:08.995954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ff316-eab3-4d3c-9b71-8b6ce51fd5d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:08.997565 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fc5ff316-eab3-4d3c-9b71-8b6ce51fd5d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:09.130202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ff316-eab3-4d3c-9b71-8b6ce51fd5d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:09.131796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ff316-eab3-4d3c-9b71-8b6ce51fd5d3 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:09.154168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 748/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:11.786054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5bf7-1b67-4b66-b971-b7623acc2670 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:11.786681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5bf7-1b67-4b66-b971-b7623acc2670 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:11.786681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5bf7-1b67-4b66-b971-b7623acc2670 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:11.787869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 714/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:11.793451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664fa00-2795-4627-b98d-7269c428bd52 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:11.793955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664fa00-2795-4627-b98d-7269c428bd52 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:11.794057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664fa00-2795-4627-b98d-7269c428bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:11.867306 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6664fa00-2795-4627-b98d-7269c428bd52 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:11.869364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6664fa00-2795-4627-b98d-7269c428bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:11.911284 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:11.972807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 749/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:12.258402 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a5bf7-1b67-4b66-b971-b7623acc2670 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:12.259961 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a29a5bf7-1b67-4b66-b971-b7623acc2670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:12.385827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a5bf7-1b67-4b66-b971-b7623acc2670 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:12.426111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:12.465610 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42547e0-eec2-4f3c-8fc2-8c69eb2be6db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:12.466212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42547e0-eec2-4f3c-8fc2-8c69eb2be6db service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:12.466514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42547e0-eec2-4f3c-8fc2-8c69eb2be6db service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:12.466785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42547e0-eec2-4f3c-8fc2-8c69eb2be6db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:12.468388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a42547e0-eec2-4f3c-8fc2-8c69eb2be6db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:12.583376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42547e0-eec2-4f3c-8fc2-8c69eb2be6db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:12.584742 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42547e0-eec2-4f3c-8fc2-8c69eb2be6db service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:12.604610 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 715/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:14.599030 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416873de-8ef8-4bac-b71b-5fc368543482 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:14.599641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416873de-8ef8-4bac-b71b-5fc368543482 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:14.599641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416873de-8ef8-4bac-b71b-5fc368543482 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:14.601038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 750/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:14.607144 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542b49e-1fe3-41a1-bc99-183e80b9d602 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:14.607673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542b49e-1fe3-41a1-bc99-183e80b9d602 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:14.607673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542b49e-1fe3-41a1-bc99-183e80b9d602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:14.687671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-8542b49e-1fe3-41a1-bc99-183e80b9d602 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:14.688948 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8542b49e-1fe3-41a1-bc99-183e80b9d602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:14.711600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:14.802523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 716/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:15.093260 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416873de-8ef8-4bac-b71b-5fc368543482 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:15.094843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-416873de-8ef8-4bac-b71b-5fc368543482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:15.220754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416873de-8ef8-4bac-b71b-5fc368543482 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:15.289489 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:15.296616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b12e9-10cb-4987-8334-2bd0548e842c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:15.297237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b12e9-10cb-4987-8334-2bd0548e842c service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:15.297516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b12e9-10cb-4987-8334-2bd0548e842c service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:15.297789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b12e9-10cb-4987-8334-2bd0548e842c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:15.300058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-681b12e9-10cb-4987-8334-2bd0548e842c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:15.419375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b12e9-10cb-4987-8334-2bd0548e842c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:15.420827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b12e9-10cb-4987-8334-2bd0548e842c service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:15.441578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 751/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:17.559411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c55ec6-c152-4521-8028-e6c85d1d0ade None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:17.559976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c55ec6-c152-4521-8028-e6c85d1d0ade None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:17.559976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c55ec6-c152-4521-8028-e6c85d1d0ade None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:17.561203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 717/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:45:17.567707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08eb9b-b9dc-4078-957d-645466c76233 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:17.568239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08eb9b-b9dc-4078-957d-645466c76233 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:17.568239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08eb9b-b9dc-4078-957d-645466c76233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:17.641744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ca08eb9b-b9dc-4078-957d-645466c76233 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:17.643674 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ca08eb9b-b9dc-4078-957d-645466c76233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:17.703827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:17.756741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 752/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:18.048347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c55ec6-c152-4521-8028-e6c85d1d0ade None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:18.050739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f8c55ec6-c152-4521-8028-e6c85d1d0ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:18.182950 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c55ec6-c152-4521-8028-e6c85d1d0ade None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:18.235234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:18.264553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f62b33-ff33-4867-a87a-61c0ba60db2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:18.265174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f62b33-ff33-4867-a87a-61c0ba60db2c service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:18.265453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f62b33-ff33-4867-a87a-61c0ba60db2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:18.265729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f62b33-ff33-4867-a87a-61c0ba60db2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:18.267521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-28f62b33-ff33-4867-a87a-61c0ba60db2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:18.402513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f62b33-ff33-4867-a87a-61c0ba60db2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:18.403966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f62b33-ff33-4867-a87a-61c0ba60db2c service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:18.426663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 718/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:27.754868 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=22 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:28.423866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=66 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:39.083125 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52cf7f-edf5-4852-8e95-439caf2b24a3 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:39.083693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52cf7f-edf5-4852-8e95-439caf2b24a3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:39.083859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52cf7f-edf5-4852-8e95-439caf2b24a3 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:39.085454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 753/1471] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:45:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:39.091604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0995f-3a82-4961-bce4-d62252e5534c None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:39.092146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0995f-3a82-4961-bce4-d62252e5534c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:39.092146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0995f-3a82-4961-bce4-d62252e5534c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:39.169475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7ac0995f-3a82-4961-bce4-d62252e5534c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:39.170720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7ac0995f-3a82-4961-bce4-d62252e5534c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:39.272903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 719/1472] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:39.576308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0620b5-b58f-475d-a9e6-6b42c487d2b7 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:39.576597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0620b5-b58f-475d-a9e6-6b42c487d2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:39.576869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0620b5-b58f-475d-a9e6-6b42c487d2b7 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:39.578520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 754/1473] 10.222.0.58 () {54 vars in 967 bytes} [Mon Aug 29 23:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:39.584229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ab64f-7570-4325-ada8-70644025d65a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:39.584511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ab64f-7570-4325-ada8-70644025d65a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:39.584781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ab64f-7570-4325-ada8-70644025d65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:39.661604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-530ab64f-7570-4325-ada8-70644025d65a None None] MFA Rules not processed for user `d7a59337d20e4678b8a16ac35bffbdb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:39.662785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-530ab64f-7570-4325-ada8-70644025d65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:39.684869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:39.744130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 720/1474] 10.222.0.58 () {58 vars in 1064 bytes} [Mon Aug 29 23:45:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:39.751594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0620b5-b58f-475d-a9e6-6b42c487d2b7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:39.753651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6b0620b5-b58f-475d-a9e6-6b42c487d2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:39.860517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0620b5-b58f-475d-a9e6-6b42c487d2b7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:39.936499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409cad7-d2b7-41c7-aa61-f374cd338342 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a59337d20e4678b8a16ac35bffbdb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:39.937018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409cad7-d2b7-41c7-aa61-f374cd338342 service glance] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:39.937301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409cad7-d2b7-41c7-aa61-f374cd338342 service glance] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:39.937576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409cad7-d2b7-41c7-aa61-f374cd338342 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:39.939211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0409cad7-d2b7-41c7-aa61-f374cd338342 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:39.995477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:40.052375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409cad7-d2b7-41c7-aa61-f374cd338342 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:40.053822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409cad7-d2b7-41c7-aa61-f374cd338342 service glance] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:40.073195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 755/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:42.150407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b95cef-b49c-43e0-8e0a-6730ee07edee None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:42.150966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b95cef-b49c-43e0-8e0a-6730ee07edee None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:42.150966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b95cef-b49c-43e0-8e0a-6730ee07edee None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:42.152250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 721/1476] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:42.157788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f538590-6d92-488b-94b6-3735a4d5486b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:42.158209 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f538590-6d92-488b-94b6-3735a4d5486b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:42.158316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f538590-6d92-488b-94b6-3735a4d5486b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:42.226534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6f538590-6d92-488b-94b6-3735a4d5486b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:42.227774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6f538590-6d92-488b-94b6-3735a4d5486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:42.328268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 756/1477] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:45:42.606558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b95cef-b49c-43e0-8e0a-6730ee07edee None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:42.608119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d3b95cef-b49c-43e0-8e0a-6730ee07edee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:42.735653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b95cef-b49c-43e0-8e0a-6730ee07edee None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:42.784805 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:42.829952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7caf4-dd4c-463a-9a95-15178411460e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:42.831197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7caf4-dd4c-463a-9a95-15178411460e admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:42.831379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7caf4-dd4c-463a-9a95-15178411460e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:42.831632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7caf4-dd4c-463a-9a95-15178411460e admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:42.832345 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7caf4-dd4c-463a-9a95-15178411460e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:42.834152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: WARNING py.warnings [None req-19e7caf4-dd4c-463a-9a95-15178411460e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:45:42.834152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: warnings.warn(msg) Aug 29 23:45:42.834152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: Aug 29 23:45:42.835382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7caf4-dd4c-463a-9a95-15178411460e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:42.855418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 722/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Aug 29 23:45:42 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:45:44.881019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae65d8-7ea7-49d7-871e-be33844803f3 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:44.881604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae65d8-7ea7-49d7-871e-be33844803f3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:44.881604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae65d8-7ea7-49d7-871e-be33844803f3 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:44.882941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 757/1479] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:44.888760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea4b47-bc07-4738-b036-5356e536e103 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:44.889280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea4b47-bc07-4738-b036-5356e536e103 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:44.889396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea4b47-bc07-4738-b036-5356e536e103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:44.959492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-6cea4b47-bc07-4738-b036-5356e536e103 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:44.960737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6cea4b47-bc07-4738-b036-5356e536e103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:45.058009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 723/1480] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:45.354970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae65d8-7ea7-49d7-871e-be33844803f3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:45.356495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-27ae65d8-7ea7-49d7-871e-be33844803f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:45.367224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:45.515051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae65d8-7ea7-49d7-871e-be33844803f3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:45.602013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:45.636034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c93881-37fc-493d-a5b1-7e647d4647fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:45.636655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c93881-37fc-493d-a5b1-7e647d4647fb service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:45.636935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c93881-37fc-493d-a5b1-7e647d4647fb service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:45.637204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c93881-37fc-493d-a5b1-7e647d4647fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:45.638842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-93c93881-37fc-493d-a5b1-7e647d4647fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:45.739382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c93881-37fc-493d-a5b1-7e647d4647fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:45.741077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c93881-37fc-493d-a5b1-7e647d4647fb service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:45.759826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 758/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:47.945982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fadc1d-b46c-4fe3-88bd-d788aad05b0b None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:47.946611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fadc1d-b46c-4fe3-88bd-d788aad05b0b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:47.946611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fadc1d-b46c-4fe3-88bd-d788aad05b0b None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:47.947860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 724/1482] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:47.953438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f839ec9-1e8a-4da2-83d7-9db2e11c65c0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:47.953931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f839ec9-1e8a-4da2-83d7-9db2e11c65c0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:47.954123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f839ec9-1e8a-4da2-83d7-9db2e11c65c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:48.033469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6f839ec9-1e8a-4da2-83d7-9db2e11c65c0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:48.035451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6f839ec9-1e8a-4da2-83d7-9db2e11c65c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:48.083337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:48.155143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 759/1483] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:48.447393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fadc1d-b46c-4fe3-88bd-d788aad05b0b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:48.448975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-46fadc1d-b46c-4fe3-88bd-d788aad05b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:48.473561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:48.557768 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fadc1d-b46c-4fe3-88bd-d788aad05b0b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:48.634317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d94b0e-3b5d-4b74-8f69-42dd9eb4fbd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:48.634964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d94b0e-3b5d-4b74-8f69-42dd9eb4fbd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:48.635247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d94b0e-3b5d-4b74-8f69-42dd9eb4fbd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:48.635538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d94b0e-3b5d-4b74-8f69-42dd9eb4fbd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:48.637193 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a5d94b0e-3b5d-4b74-8f69-42dd9eb4fbd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:48.835021 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:48.871155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d94b0e-3b5d-4b74-8f69-42dd9eb4fbd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:48.872646 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d94b0e-3b5d-4b74-8f69-42dd9eb4fbd3 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:48.891509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 725/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:50.925295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa75620-8261-4919-a717-efc870ef0c28 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:50.925866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa75620-8261-4919-a717-efc870ef0c28 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:50.925866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa75620-8261-4919-a717-efc870ef0c28 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:50.927197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 760/1485] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:50.932620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8656db-6043-4144-a1f3-b6b3b9f54769 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:50.933429 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8656db-6043-4144-a1f3-b6b3b9f54769 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:50.933429 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8656db-6043-4144-a1f3-b6b3b9f54769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:51.002043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-bc8656db-6043-4144-a1f3-b6b3b9f54769 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:51.003914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bc8656db-6043-4144-a1f3-b6b3b9f54769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:51.106335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 726/1486] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:51.396451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa75620-8261-4919-a717-efc870ef0c28 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:51.397990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7aa75620-8261-4919-a717-efc870ef0c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:51.623946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa75620-8261-4919-a717-efc870ef0c28 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:51.697548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:51.699367 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fe6ac-3fed-4b3c-909a-6a4ab4f8793e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:51.699969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fe6ac-3fed-4b3c-909a-6a4ab4f8793e service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:51.700253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fe6ac-3fed-4b3c-909a-6a4ab4f8793e service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:51.700524 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fe6ac-3fed-4b3c-909a-6a4ab4f8793e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:51.702114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3b1fe6ac-3fed-4b3c-909a-6a4ab4f8793e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:51.814010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fe6ac-3fed-4b3c-909a-6a4ab4f8793e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:51.816134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fe6ac-3fed-4b3c-909a-6a4ab4f8793e service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:51.834649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 761/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:53.939275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db41f8-0dcc-4484-85ed-5804d7de3fa9 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:53.939897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db41f8-0dcc-4484-85ed-5804d7de3fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:53.939897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db41f8-0dcc-4484-85ed-5804d7de3fa9 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:53.941647 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 727/1488] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:53.947954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bd9cc-0fbc-41e1-a6b4-2cf57cdfa37b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:53.948480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bd9cc-0fbc-41e1-a6b4-2cf57cdfa37b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:53.948948 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bd9cc-0fbc-41e1-a6b4-2cf57cdfa37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:54.019636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-988bd9cc-0fbc-41e1-a6b4-2cf57cdfa37b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:54.020854 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-988bd9cc-0fbc-41e1-a6b4-2cf57cdfa37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:54.075232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:54.127190 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 762/1489] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:54.425647 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db41f8-0dcc-4484-85ed-5804d7de3fa9 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:54.427221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e7db41f8-0dcc-4484-85ed-5804d7de3fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:54.463494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:54.533316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db41f8-0dcc-4484-85ed-5804d7de3fa9 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:54.614054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03a97c-33a2-42ca-8e64-e89adea89697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:54.614575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03a97c-33a2-42ca-8e64-e89adea89697 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:54.614911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03a97c-33a2-42ca-8e64-e89adea89697 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:54.615182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03a97c-33a2-42ca-8e64-e89adea89697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:54.617174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-dc03a97c-33a2-42ca-8e64-e89adea89697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:54.700249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:54.731229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03a97c-33a2-42ca-8e64-e89adea89697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:54.732667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03a97c-33a2-42ca-8e64-e89adea89697 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:54.751395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 728/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:45:57.081507 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f4f44-0597-495d-983d-5b2fdb91f0ea None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:57.082081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f4f44-0597-495d-983d-5b2fdb91f0ea None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:57.082081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f4f44-0597-495d-983d-5b2fdb91f0ea None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:57.083547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 763/1491] 10.222.0.58 () {54 vars in 927 bytes} [Mon Aug 29 23:45:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:45:57.089009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe191af-626f-453f-9531-8546253fa9ed None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:57.089458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe191af-626f-453f-9531-8546253fa9ed None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:57.089564 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe191af-626f-453f-9531-8546253fa9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:57.157221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-3fe191af-626f-453f-9531-8546253fa9ed None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:45:57.158448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3fe191af-626f-453f-9531-8546253fa9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:57.266872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 729/1492] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Aug 29 23:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:45:57.558364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f4f44-0597-495d-983d-5b2fdb91f0ea None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:45:57.559927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-964f4f44-0597-495d-983d-5b2fdb91f0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:57.670757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f4f44-0597-495d-983d-5b2fdb91f0ea None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:45:57.751174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3e8c7-a6e6-4b93-a460-375414c9d7b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:45:57.751774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3e8c7-a6e6-4b93-a460-375414c9d7b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:45:57.752043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3e8c7-a6e6-4b93-a460-375414c9d7b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:45:57.752319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3e8c7-a6e6-4b93-a460-375414c9d7b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:45:57.754133 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c1a3e8c7-a6e6-4b93-a460-375414c9d7b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:45:57.763536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:45:57.869024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3e8c7-a6e6-4b93-a460-375414c9d7b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:45:57.870515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3e8c7-a6e6-4b93-a460-375414c9d7b3 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:45:57.894963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 764/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:45:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:46:07.264936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=82 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:46:07.890833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=50 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:24.130997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47067099-83c1-4d02-87f7-bc0515482385 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:24.131697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47067099-83c1-4d02-87f7-bc0515482385 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:24.131697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47067099-83c1-4d02-87f7-bc0515482385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:24.209102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-47067099-83c1-4d02-87f7-bc0515482385 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:47:24.210259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-47067099-83c1-4d02-87f7-bc0515482385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:24.307428 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 730/1494] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:47:24.315619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a4bbb-1e65-4e65-8be1-bba1eaeda274 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:24.316133 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a4bbb-1e65-4e65-8be1-bba1eaeda274 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:24.316133 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a4bbb-1e65-4e65-8be1-bba1eaeda274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:24.394670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e83a4bbb-1e65-4e65-8be1-bba1eaeda274 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:47:24.396616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e83a4bbb-1e65-4e65-8be1-bba1eaeda274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:24.491958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 765/1495] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:47:24.496528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47067099-83c1-4d02-87f7-bc0515482385 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:47:24.497771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-47067099-83c1-4d02-87f7-bc0515482385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:24.509687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47067099-83c1-4d02-87f7-bc0515482385 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:47:24.571629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:24.606703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e358b5c-a811-45f7-a82d-cef6b372642a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:47:24.607304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e358b5c-a811-45f7-a82d-cef6b372642a admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:24.607576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e358b5c-a811-45f7-a82d-cef6b372642a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:24.607842 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e358b5c-a811-45f7-a82d-cef6b372642a admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:24.608481 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e358b5c-a811-45f7-a82d-cef6b372642a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:47:24.608779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e358b5c-a811-45f7-a82d-cef6b372642a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:47:24.609785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: WARNING py.warnings [None req-6e358b5c-a811-45f7-a82d-cef6b372642a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:47:24.609785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: warnings.warn(msg) Aug 29 23:47:24.609785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: Aug 29 23:47:24.610699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e358b5c-a811-45f7-a82d-cef6b372642a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:47:24.622083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 731/1496] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:47:24.631065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a4bbb-1e65-4e65-8be1-bba1eaeda274 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:47:24.640118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a4bbb-1e65-4e65-8be1-bba1eaeda274 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:47:24.702148 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:24.736498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c06d3-cd96-49ea-8ba0-9a32b55d6132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:47:24.737101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c06d3-cd96-49ea-8ba0-9a32b55d6132 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:24.737375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c06d3-cd96-49ea-8ba0-9a32b55d6132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:24.737650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c06d3-cd96-49ea-8ba0-9a32b55d6132 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:24.738690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c06d3-cd96-49ea-8ba0-9a32b55d6132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:47:24.741264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-713c06d3-cd96-49ea-8ba0-9a32b55d6132 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:47:24.741264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 29 23:47:24.741264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: Aug 29 23:47:24.742542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c06d3-cd96-49ea-8ba0-9a32b55d6132 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:47:24.872218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 766/1497] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:47:24 2022] POST /identity/v3/projects => generated 379 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:47:24.876679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e358b5c-a811-45f7-a82d-cef6b372642a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:47:24.878268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6e358b5c-a811-45f7-a82d-cef6b372642a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:24.922162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e358b5c-a811-45f7-a82d-cef6b372642a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:47:25.016898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3cb25-23aa-415a-80ee-5bb5fa8bc8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:47:25.017426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3cb25-23aa-415a-80ee-5bb5fa8bc8af admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:25.017696 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3cb25-23aa-415a-80ee-5bb5fa8bc8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:25.017966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3cb25-23aa-415a-80ee-5bb5fa8bc8af admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:25.018633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3cb25-23aa-415a-80ee-5bb5fa8bc8af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:47:25.019653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: WARNING py.warnings [None req-e4d3cb25-23aa-415a-80ee-5bb5fa8bc8af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:47:25.019653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: warnings.warn(msg) Aug 29 23:47:25.019653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: Aug 29 23:47:25.021373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3cb25-23aa-415a-80ee-5bb5fa8bc8af admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:47:25.111203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 732/1498] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:47:24 2022] POST /identity/v3/users => generated 290 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:47:25.115670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c06d3-cd96-49ea-8ba0-9a32b55d6132 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:47:25.117184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-713c06d3-cd96-49ea-8ba0-9a32b55d6132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:25.180445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c06d3-cd96-49ea-8ba0-9a32b55d6132 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:47:25.265394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:25.278025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18694947-9121-4d1d-a0f1-6f50486cda17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:47:25.278025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18694947-9121-4d1d-a0f1-6f50486cda17 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:25.278025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18694947-9121-4d1d-a0f1-6f50486cda17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:25.278025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18694947-9121-4d1d-a0f1-6f50486cda17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:25.278498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18694947-9121-4d1d-a0f1-6f50486cda17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:47:25.278923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-18694947-9121-4d1d-a0f1-6f50486cda17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:47:25.278923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 29 23:47:25.278923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: Aug 29 23:47:25.279798 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18694947-9121-4d1d-a0f1-6f50486cda17 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:47:25.293066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 767/1499] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:47:25.297243 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3cb25-23aa-415a-80ee-5bb5fa8bc8af admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:47:25.298830 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e4d3cb25-23aa-415a-80ee-5bb5fa8bc8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:25.310813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3cb25-23aa-415a-80ee-5bb5fa8bc8af admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:47:25.337732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:25.407503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8bfbb-ac1d-4a7d-b9e7-e37a9577a5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:47:25.408107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8bfbb-ac1d-4a7d-b9e7-e37a9577a5bc admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:25.408385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8bfbb-ac1d-4a7d-b9e7-e37a9577a5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:25.408653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8bfbb-ac1d-4a7d-b9e7-e37a9577a5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:25.409287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8bfbb-ac1d-4a7d-b9e7-e37a9577a5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:47:25.410296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: WARNING py.warnings [None req-beb8bfbb-ac1d-4a7d-b9e7-e37a9577a5bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:47:25.410296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: warnings.warn(msg) Aug 29 23:47:25.410296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: Aug 29 23:47:25.411792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8bfbb-ac1d-4a7d-b9e7-e37a9577a5bc admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:47:25.428335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 733/1500] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:47:25.432782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18694947-9121-4d1d-a0f1-6f50486cda17 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:47:25.434323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-18694947-9121-4d1d-a0f1-6f50486cda17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:25.468021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18694947-9121-4d1d-a0f1-6f50486cda17 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:47:25.566260 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4069f7-425c-4bbd-8d12-22bda5154624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:47:25.566877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4069f7-425c-4bbd-8d12-22bda5154624 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:25.567224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4069f7-425c-4bbd-8d12-22bda5154624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:25.568003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4069f7-425c-4bbd-8d12-22bda5154624 admin admin] PATH_INFO: `/v3/projects/12f0197263074ceaac95b9f7e27a88cc/users/79c1879b53004b57899a7fa7c9c26323/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:25.590730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4069f7-425c-4bbd-8d12-22bda5154624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f0197263074ceaac95b9f7e27a88cc, user_id=79c1879b53004b57899a7fa7c9c26323, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:47:25.591791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-2f4069f7-425c-4bbd-8d12-22bda5154624 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:47:25.591791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 29 23:47:25.591791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: Aug 29 23:47:25.593729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4069f7-425c-4bbd-8d12-22bda5154624 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:47:25.687642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 768/1501] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:47:25 2022] PUT /identity/v3/projects/12f0197263074ceaac95b9f7e27a88cc/users/79c1879b53004b57899a7fa7c9c26323/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:47:25.701002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da6ec5-359a-4d1f-8827-b858434f07d7 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:25.701289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da6ec5-359a-4d1f-8827-b858434f07d7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:25.701566 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da6ec5-359a-4d1f-8827-b858434f07d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:25.759634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d6da6ec5-359a-4d1f-8827-b858434f07d7 None None] MFA Rules not processed for user `79c1879b53004b57899a7fa7c9c26323`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:47:25.760751 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d6da6ec5-359a-4d1f-8827-b858434f07d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:25.842052 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 734/1502] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:47:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:47:25.855842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d006d-86b4-441a-bd8c-82f374820129 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:25.856129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d006d-86b4-441a-bd8c-82f374820129 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:25.856447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d006d-86b4-441a-bd8c-82f374820129 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:25.857802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 769/1503] 10.222.0.58 () {54 vars in 964 bytes} [Mon Aug 29 23:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:47:25.863018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b19657-8e7b-4743-ad95-cdf6832089c9 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:25.863304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b19657-8e7b-4743-ad95-cdf6832089c9 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:25.863581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b19657-8e7b-4743-ad95-cdf6832089c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:25.897306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:25.943498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-a2b19657-8e7b-4743-ad95-cdf6832089c9 None None] MFA Rules not processed for user `64a42591a6814a0cbcb8444829cb25b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:47:25.944858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a2b19657-8e7b-4743-ad95-cdf6832089c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:26.022154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 735/1504] 10.222.0.58 () {58 vars in 1061 bytes} [Mon Aug 29 23:47:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:47:26.030056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d006d-86b4-441a-bd8c-82f374820129 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:47:26.031725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fb3d006d-86b4-441a-bd8c-82f374820129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:26.084562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:26.138928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d006d-86b4-441a-bd8c-82f374820129 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:47:26.216333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67621d41-0338-4319-914e-2b9ac41e6fac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:47:26.216945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67621d41-0338-4319-914e-2b9ac41e6fac service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:26.217224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67621d41-0338-4319-914e-2b9ac41e6fac service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:26.217502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67621d41-0338-4319-914e-2b9ac41e6fac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:26.219144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-67621d41-0338-4319-914e-2b9ac41e6fac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:26.325857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67621d41-0338-4319-914e-2b9ac41e6fac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:47:26.327329 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67621d41-0338-4319-914e-2b9ac41e6fac service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:47:26.330796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:26.345858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 770/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:47:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:47:26.392379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef1c6c-4be1-4eb4-b600-f91982e822ee None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:26.392842 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef1c6c-4be1-4eb4-b600-f91982e822ee None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:26.392978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef1c6c-4be1-4eb4-b600-f91982e822ee None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:26.394241 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 736/1506] 10.222.0.58 () {52 vars in 830 bytes} [Mon Aug 29 23:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:47:26.398284 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67621d41-0338-4319-914e-2b9ac41e6fac service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:47:26.399542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-67621d41-0338-4319-914e-2b9ac41e6fac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:26.500012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67621d41-0338-4319-914e-2b9ac41e6fac service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:47:26.595402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:47:26.596084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:26.596308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:26.596575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] PATH_INFO: `/v3/users/79c1879b53004b57899a7fa7c9c26323` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:26.609525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c1879b53004b57899a7fa7c9c26323)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:47:26.610619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:47:26.610619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 29 23:47:26.610619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: Aug 29 23:47:26.611843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:47:26.622742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:26.832712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c1879b53004b57899a7fa7c9c26323'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:47:26.856313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c1879b53004b57899a7fa7c9c26323'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:47:26.871093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c1879b53004b57899a7fa7c9c26323'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:47:26.944903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c1879b53004b57899a7fa7c9c26323'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:47:26.945495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c1879b53004b57899a7fa7c9c26323'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:47:26.952107 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 771/1507] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:47:26 2022] DELETE /identity/v3/users/79c1879b53004b57899a7fa7c9c26323 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:47:26.956604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791345-b5f1-46d5-aa8f-94788018a721 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:26.956890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791345-b5f1-46d5-aa8f-94788018a721 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:26.957164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791345-b5f1-46d5-aa8f-94788018a721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:27.019906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-6f791345-b5f1-46d5-aa8f-94788018a721 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:47:27.021664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6f791345-b5f1-46d5-aa8f-94788018a721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:27.143838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:27.156841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 737/1508] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:47:27.167779 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:47:27.169061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:27.272517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76b709-1631-439a-98d7-a0e0fee51134 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:47:27.331137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:27.352378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2cc2b-0d6d-452c-9f93-b91861393f79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:47:27.353021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2cc2b-0d6d-452c-9f93-b91861393f79 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:27.353296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2cc2b-0d6d-452c-9f93-b91861393f79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:27.353565 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2cc2b-0d6d-452c-9f93-b91861393f79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:27.355310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d1f2cc2b-0d6d-452c-9f93-b91861393f79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:27.466494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2cc2b-0d6d-452c-9f93-b91861393f79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:47:27.467874 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2cc2b-0d6d-452c-9f93-b91861393f79 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:47:27.486402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 772/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:47:28.050557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f791345-b5f1-46d5-aa8f-94788018a721 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:47:28.051889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6f791345-b5f1-46d5-aa8f-94788018a721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:47:28.063948 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f791345-b5f1-46d5-aa8f-94788018a721 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:47:28.170295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:47:28.170960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:47:28.171239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:47:28.171517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] PATH_INFO: `/v3/projects/12f0197263074ceaac95b9f7e27a88cc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:47:28.182290 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] RBAC: Authorizing `identity:delete_project(project_id=12f0197263074ceaac95b9f7e27a88cc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:47:28.183363 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: WARNING py.warnings [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:47:28.183363 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: warnings.warn(msg) Aug 29 23:47:28.183363 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: Aug 29 23:47:28.184570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:47:28.372979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12f0197263074ceaac95b9f7e27a88cc', 'request_id': 'req-1d0e231e-e6ea-4667-af1a-3e3272d256df'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:47:28.430197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12f0197263074ceaac95b9f7e27a88cc', 'request_id': 'req-1d0e231e-e6ea-4667-af1a-3e3272d256df'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:47:28.437441 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12f0197263074ceaac95b9f7e27a88cc', 'request_id': 'req-1d0e231e-e6ea-4667-af1a-3e3272d256df'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:47:28.470139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] The token cache is being invalidate because project 12f0197263074ceaac95b9f7e27a88cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:47:28.470455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:47:28.470977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1d0e231e-e6ea-4667-af1a-3e3272d256df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:47:28.472278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 738/1510] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:47:28 2022] DELETE /identity/v3/projects/12f0197263074ceaac95b9f7e27a88cc => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:47:37.556676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=62 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:47:38.468017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:49:01.503270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c185c02-fc25-42c3-a7e2-9bcbc2b57e68 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:01.503872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c185c02-fc25-42c3-a7e2-9bcbc2b57e68 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:01.503872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c185c02-fc25-42c3-a7e2-9bcbc2b57e68 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:01.505145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 773/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:49:01.509939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d20d6b-1442-4677-a38a-d7d0461ac977 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:01.510408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d20d6b-1442-4677-a38a-d7d0461ac977 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:01.510541 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d20d6b-1442-4677-a38a-d7d0461ac977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:01.593640 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-00d20d6b-1442-4677-a38a-d7d0461ac977 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:49:01.594894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-00d20d6b-1442-4677-a38a-d7d0461ac977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:01.697805 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 739/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:49:01.854236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c185c02-fc25-42c3-a7e2-9bcbc2b57e68 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:49:01.857400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2c185c02-fc25-42c3-a7e2-9bcbc2b57e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:01.983705 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c185c02-fc25-42c3-a7e2-9bcbc2b57e68 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:49:01.989245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e0ca0-f710-4ad1-b696-6f3f2a5c6236 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:49:01.989860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e0ca0-f710-4ad1-b696-6f3f2a5c6236 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:01.990159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e0ca0-f710-4ad1-b696-6f3f2a5c6236 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:01.990459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e0ca0-f710-4ad1-b696-6f3f2a5c6236 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:01.999203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e0ca0-f710-4ad1-b696-6f3f2a5c6236 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:49:02.000801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e0ca0-f710-4ad1-b696-6f3f2a5c6236 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:49:02.010290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 774/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:49:02.852907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13299-dd81-4f60-897c-072bbd97e923 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:02.853481 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13299-dd81-4f60-897c-072bbd97e923 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:02.853481 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13299-dd81-4f60-897c-072bbd97e923 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:02.854811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 740/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:49:02.860801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffb562-3bfd-41f2-8625-a991441c99f0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:02.861279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffb562-3bfd-41f2-8625-a991441c99f0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:02.861396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffb562-3bfd-41f2-8625-a991441c99f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:02.881624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5bffb562-3bfd-41f2-8625-a991441c99f0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:49:02.882886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5bffb562-3bfd-41f2-8625-a991441c99f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:02.891039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 775/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:49:03.045474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a13299-dd81-4f60-897c-072bbd97e923 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:49:03.060600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a13299-dd81-4f60-897c-072bbd97e923 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:49:03.066118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af649e-9cdd-4524-afd1-99bfd82843f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:49:03.066760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af649e-9cdd-4524-afd1-99bfd82843f6 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:03.067042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af649e-9cdd-4524-afd1-99bfd82843f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:03.067312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af649e-9cdd-4524-afd1-99bfd82843f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:03.077452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af649e-9cdd-4524-afd1-99bfd82843f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:49:03.079176 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af649e-9cdd-4524-afd1-99bfd82843f6 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:49:03.089685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 741/1516] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:49:03.511137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdf373-69cb-4e4b-81e4-05e3ebbd9815 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:03.511413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdf373-69cb-4e4b-81e4-05e3ebbd9815 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:03.511739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdf373-69cb-4e4b-81e4-05e3ebbd9815 None None] PATH_INFO: `/v3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:03.513393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 776/1517] 10.222.0.61 () {54 vars in 925 bytes} [Mon Aug 29 23:49:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:49:03.521787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bda24-acf1-415c-b91f-137caf4d265e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:03.522073 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bda24-acf1-415c-b91f-137caf4d265e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:03.522354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bda24-acf1-415c-b91f-137caf4d265e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:03.581664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-9a8bda24-acf1-415c-b91f-137caf4d265e None None] MFA Rules not processed for user `fb770d4379e74bf79ad4280329c771c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:49:03.582897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9a8bda24-acf1-415c-b91f-137caf4d265e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:03.601256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:49:03.655581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 742/1518] 10.222.0.61 () {58 vars in 1015 bytes} [Mon Aug 29 23:49:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:49:07.828482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e0d18-0d42-4206-8966-426b9ec5d9e5 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:07.829047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e0d18-0d42-4206-8966-426b9ec5d9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:07.829047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e0d18-0d42-4206-8966-426b9ec5d9e5 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:07.830452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 777/1519] 10.222.0.58 () {54 vars in 961 bytes} [Mon Aug 29 23:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:49:07.835715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c7d5e-2d55-4f39-98bd-532376e96565 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:07.836241 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c7d5e-2d55-4f39-98bd-532376e96565 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:07.836241 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c7d5e-2d55-4f39-98bd-532376e96565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:07.857179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-a87c7d5e-2d55-4f39-98bd-532376e96565 None None] MFA Rules not processed for user `fb770d4379e74bf79ad4280329c771c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:49:07.858380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a87c7d5e-2d55-4f39-98bd-532376e96565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:07.866785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 743/1520] 10.222.0.58 () {58 vars in 1058 bytes} [Mon Aug 29 23:49:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:49:07.874837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e0d18-0d42-4206-8966-426b9ec5d9e5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:49:07.888429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e0d18-0d42-4206-8966-426b9ec5d9e5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:49:07.893463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d7918-dc37-4688-9ee7-992edafd9183 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:49:07.894068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d7918-dc37-4688-9ee7-992edafd9183 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:07.894345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d7918-dc37-4688-9ee7-992edafd9183 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:07.894653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d7918-dc37-4688-9ee7-992edafd9183 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:07.904218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d7918-dc37-4688-9ee7-992edafd9183 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:49:07.905825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d7918-dc37-4688-9ee7-992edafd9183 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:49:07.915593 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 778/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:49:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:49:14.322035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc742db-1954-428a-aec5-8353d4bdced4 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:14.322873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc742db-1954-428a-aec5-8353d4bdced4 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:14.322873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc742db-1954-428a-aec5-8353d4bdced4 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:14.325051 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 744/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:49:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:49:14.330378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e8235-d266-45f7-a7cf-b341f5999a3c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:14.330855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e8235-d266-45f7-a7cf-b341f5999a3c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:14.330961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e8235-d266-45f7-a7cf-b341f5999a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:14.351520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ba8e8235-d266-45f7-a7cf-b341f5999a3c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:49:14.352730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ba8e8235-d266-45f7-a7cf-b341f5999a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:14.360969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 779/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:49:14.516752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc742db-1954-428a-aec5-8353d4bdced4 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:49:14.531144 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc742db-1954-428a-aec5-8353d4bdced4 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:49:14.536756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec62369-8145-4342-a669-b37c2eb141f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:49:14.537354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec62369-8145-4342-a669-b37c2eb141f0 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:14.537669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec62369-8145-4342-a669-b37c2eb141f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:14.537940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec62369-8145-4342-a669-b37c2eb141f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:14.546474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec62369-8145-4342-a669-b37c2eb141f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:49:14.548183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec62369-8145-4342-a669-b37c2eb141f0 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:49:14.557497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 745/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:49:15.392506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4121db4-2863-4989-9c21-8a081863332f None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:15.393057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4121db4-2863-4989-9c21-8a081863332f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:15.393057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4121db4-2863-4989-9c21-8a081863332f None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:15.394398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 780/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 23:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:49:15.399373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55056bc-9998-4907-8850-8978344b723d None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:15.399795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55056bc-9998-4907-8850-8978344b723d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:15.399928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55056bc-9998-4907-8850-8978344b723d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:15.419180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d55056bc-9998-4907-8850-8978344b723d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:49:15.420377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d55056bc-9998-4907-8850-8978344b723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:15.428468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 746/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 23:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:49:15.581426 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4121db4-2863-4989-9c21-8a081863332f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:49:15.596273 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4121db4-2863-4989-9c21-8a081863332f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:49:15.602114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767808d-c354-4233-bc32-e34d80317443 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:49:15.602689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767808d-c354-4233-bc32-e34d80317443 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:15.603258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767808d-c354-4233-bc32-e34d80317443 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:15.603258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767808d-c354-4233-bc32-e34d80317443 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:15.612505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767808d-c354-4233-bc32-e34d80317443 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:49:15.614205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767808d-c354-4233-bc32-e34d80317443 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:49:15.624500 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 781/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:49:19.393864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4bd9f-7be7-40f8-8686-dceea0d2e826 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:19.394376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4bd9f-7be7-40f8-8686-dceea0d2e826 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:19.394376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4bd9f-7be7-40f8-8686-dceea0d2e826 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:19.395676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 747/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:49:19.401047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cda4a-939f-4b88-a9a8-60c69cee1f1d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:19.401480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cda4a-939f-4b88-a9a8-60c69cee1f1d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:19.401579 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cda4a-939f-4b88-a9a8-60c69cee1f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:19.422467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7b0cda4a-939f-4b88-a9a8-60c69cee1f1d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:49:19.423692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7b0cda4a-939f-4b88-a9a8-60c69cee1f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:19.431530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 782/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:49:19.734477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4bd9f-7be7-40f8-8686-dceea0d2e826 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:49:19.736401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-76c4bd9f-7be7-40f8-8686-dceea0d2e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:19.814247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4bd9f-7be7-40f8-8686-dceea0d2e826 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:49:19.817836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba1cf4-143f-4339-aef9-bafa9b9bce30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:49:19.818452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba1cf4-143f-4339-aef9-bafa9b9bce30 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:19.818735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba1cf4-143f-4339-aef9-bafa9b9bce30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:19.819012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba1cf4-143f-4339-aef9-bafa9b9bce30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:19.828987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba1cf4-143f-4339-aef9-bafa9b9bce30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:49:19.830448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba1cf4-143f-4339-aef9-bafa9b9bce30 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:49:19.839992 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 748/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:49:21.757088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0b510-945f-465b-be21-51467a75320e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:21.757656 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0b510-945f-465b-be21-51467a75320e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:21.757656 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0b510-945f-465b-be21-51467a75320e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:21.758930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 783/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:49:21.764602 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5182a5-bb0b-4d2c-b6a9-5d68865e2b6f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:21.765081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5182a5-bb0b-4d2c-b6a9-5d68865e2b6f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:21.765183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5182a5-bb0b-4d2c-b6a9-5d68865e2b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:21.786396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-2c5182a5-bb0b-4d2c-b6a9-5d68865e2b6f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:49:21.787635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2c5182a5-bb0b-4d2c-b6a9-5d68865e2b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:21.795659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 749/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:49:22.081412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0b510-945f-465b-be21-51467a75320e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:49:22.093988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0b510-945f-465b-be21-51467a75320e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:49:22.097509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d6609-fd55-48df-8209-7fbfc5a1b68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:49:22.098108 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d6609-fd55-48df-8209-7fbfc5a1b68e service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:22.098386 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d6609-fd55-48df-8209-7fbfc5a1b68e service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:22.098686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d6609-fd55-48df-8209-7fbfc5a1b68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:22.108324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d6609-fd55-48df-8209-7fbfc5a1b68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:49:22.109758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d6609-fd55-48df-8209-7fbfc5a1b68e service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:49:22.119033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 784/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:49:25.752218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a12e0-ed3f-4d77-b185-0c238f05c06d None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:25.752741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a12e0-ed3f-4d77-b185-0c238f05c06d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:25.752741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a12e0-ed3f-4d77-b185-0c238f05c06d None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:25.754005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 750/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:49:25.759122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda084d-36a1-4cfb-acaf-bdbfb8e486ee None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:25.759505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda084d-36a1-4cfb-acaf-bdbfb8e486ee None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:25.759666 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda084d-36a1-4cfb-acaf-bdbfb8e486ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:25.780141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-bcda084d-36a1-4cfb-acaf-bdbfb8e486ee None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:49:25.781327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bcda084d-36a1-4cfb-acaf-bdbfb8e486ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:25.789014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 785/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:49:26.360774 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a12e0-ed3f-4d77-b185-0c238f05c06d None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:49:26.362491 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b99a12e0-ed3f-4d77-b185-0c238f05c06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:26.428166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:49:26.441191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a12e0-ed3f-4d77-b185-0c238f05c06d None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:49:26.444766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b56f9-dfbb-417d-9702-a3be5847c475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:49:26.445359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b56f9-dfbb-417d-9702-a3be5847c475 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:26.445631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b56f9-dfbb-417d-9702-a3be5847c475 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:26.445897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b56f9-dfbb-417d-9702-a3be5847c475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:26.455332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b56f9-dfbb-417d-9702-a3be5847c475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:49:26.456893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b56f9-dfbb-417d-9702-a3be5847c475 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:49:26.467169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 751/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:49:28.501783 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e438f92-e153-4eec-b871-0f7392041ad5 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:28.502373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e438f92-e153-4eec-b871-0f7392041ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:28.502513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e438f92-e153-4eec-b871-0f7392041ad5 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:28.503808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 786/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 23:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:49:28.509920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c75b32-d082-42a3-8085-4f42008451cc None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:28.510352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c75b32-d082-42a3-8085-4f42008451cc None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:28.510557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c75b32-d082-42a3-8085-4f42008451cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:28.531239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-84c75b32-d082-42a3-8085-4f42008451cc None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:49:28.532443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-84c75b32-d082-42a3-8085-4f42008451cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:28.540212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 752/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 23:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:49:29.113684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e438f92-e153-4eec-b871-0f7392041ad5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:49:29.115341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1e438f92-e153-4eec-b871-0f7392041ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:49:29.123174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=84 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:49:29.128975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e438f92-e153-4eec-b871-0f7392041ad5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:49:29.132758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6f4bc-c7d1-4f2c-bb1c-df649d7b0713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:49:29.133358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6f4bc-c7d1-4f2c-bb1c-df649d7b0713 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:49:29.133707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6f4bc-c7d1-4f2c-bb1c-df649d7b0713 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:49:29.133979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6f4bc-c7d1-4f2c-bb1c-df649d7b0713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:49:29.144632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6f4bc-c7d1-4f2c-bb1c-df649d7b0713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:49:29.146064 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6f4bc-c7d1-4f2c-bb1c-df649d7b0713 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:49:29.158408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 787/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:49:38.525604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=11 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:49:39.160906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:50:57.256948 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dd8cb-5b92-49fd-829f-6c624beca2e1 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.258024 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dd8cb-5b92-49fd-829f-6c624beca2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.258576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dd8cb-5b92-49fd-829f-6c624beca2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.290987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-bb2dd8cb-5b92-49fd-829f-6c624beca2e1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:57.292960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bb2dd8cb-5b92-49fd-829f-6c624beca2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:57.304155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 753/1540] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:57.313345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcab5d-daf9-45ef-963f-5c7049492618 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.313935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcab5d-daf9-45ef-963f-5c7049492618 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.313935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcab5d-daf9-45ef-963f-5c7049492618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.348205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b8bcab5d-daf9-45ef-963f-5c7049492618 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:57.349625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b8bcab5d-daf9-45ef-963f-5c7049492618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:57.367410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 788/1541] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:57.372694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dd8cb-5b92-49fd-829f-6c624beca2e1 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:57.391076 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dd8cb-5b92-49fd-829f-6c624beca2e1 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:57.398457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284aff5-ffa4-4f85-a30a-bfad4f1f41ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:57.399876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284aff5-ffa4-4f85-a30a-bfad4f1f41ae admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.400360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284aff5-ffa4-4f85-a30a-bfad4f1f41ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.400823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284aff5-ffa4-4f85-a30a-bfad4f1f41ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.401697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284aff5-ffa4-4f85-a30a-bfad4f1f41ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:50:57.402197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284aff5-ffa4-4f85-a30a-bfad4f1f41ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:57.404073 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284aff5-ffa4-4f85-a30a-bfad4f1f41ae admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:57.425574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 754/1542] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:50:57.433912 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcab5d-daf9-45ef-963f-5c7049492618 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:57.448546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcab5d-daf9-45ef-963f-5c7049492618 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:57.459446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed595499-8d68-47ae-8dcc-78e9e33eddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:57.460092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed595499-8d68-47ae-8dcc-78e9e33eddf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.460375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed595499-8d68-47ae-8dcc-78e9e33eddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.460651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed595499-8d68-47ae-8dcc-78e9e33eddf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.461319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed595499-8d68-47ae-8dcc-78e9e33eddf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:57.463448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed595499-8d68-47ae-8dcc-78e9e33eddf5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:57.565198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1054102-26c7-46d6-bd75-ef92c82e8426 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.566072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1054102-26c7-46d6-bd75-ef92c82e8426 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.566608 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1054102-26c7-46d6-bd75-ef92c82e8426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.595734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-b1054102-26c7-46d6-bd75-ef92c82e8426 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:57.598299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b1054102-26c7-46d6-bd75-ef92c82e8426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:57.606884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 789/1543] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/projects => generated 363 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:57.611479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed595499-8d68-47ae-8dcc-78e9e33eddf5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:57.708084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 755/1544] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:57.718846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b0517-69c6-4d8e-97a0-33475a4509df None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.719481 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b0517-69c6-4d8e-97a0-33475a4509df None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.720347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b0517-69c6-4d8e-97a0-33475a4509df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.751551 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed595499-8d68-47ae-8dcc-78e9e33eddf5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:57.763380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-272b0517-69c6-4d8e-97a0-33475a4509df None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:57.765979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-272b0517-69c6-4d8e-97a0-33475a4509df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:57.769485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4020d4-65c9-4bb8-b764-4ed0c856c27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:57.770471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4020d4-65c9-4bb8-b764-4ed0c856c27d admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.770953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4020d4-65c9-4bb8-b764-4ed0c856c27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.771439 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4020d4-65c9-4bb8-b764-4ed0c856c27d admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.772301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4020d4-65c9-4bb8-b764-4ed0c856c27d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:57.773663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-6a4020d4-65c9-4bb8-b764-4ed0c856c27d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:50:57.773663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 29 23:50:57.773663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: Aug 29 23:50:57.776528 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4020d4-65c9-4bb8-b764-4ed0c856c27d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:57.793344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 756/1545] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:50:57.795065 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c104f60-b666-4220-b7ce-1f3667497a36 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.795785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c104f60-b666-4220-b7ce-1f3667497a36 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.796278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c104f60-b666-4220-b7ce-1f3667497a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.823892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-0c104f60-b666-4220-b7ce-1f3667497a36 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:57.825195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-0c104f60-b666-4220-b7ce-1f3667497a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:57.837794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 757/1546] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:57.837794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91597a3-f726-4081-bd21-c79c400ffad8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.837794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91597a3-f726-4081-bd21-c79c400ffad8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.837794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91597a3-f726-4081-bd21-c79c400ffad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.858297 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-a91597a3-f726-4081-bd21-c79c400ffad8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:57.859670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a91597a3-f726-4081-bd21-c79c400ffad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:57.870308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 758/1547] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:57.871550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91597a3-f726-4081-bd21-c79c400ffad8 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:57.884920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91597a3-f726-4081-bd21-c79c400ffad8 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:57.887696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 790/1548] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/users => generated 282 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:57.889977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfacccf-530b-4f1a-a712-fe6d9ff4f572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:57.890623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfacccf-530b-4f1a-a712-fe6d9ff4f572 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.895782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfacccf-530b-4f1a-a712-fe6d9ff4f572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.895782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfacccf-530b-4f1a-a712-fe6d9ff4f572 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.895782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfacccf-530b-4f1a-a712-fe6d9ff4f572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:50:57.896094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d2865-d9dc-45ea-97b4-0ec394740e84 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.896194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfacccf-530b-4f1a-a712-fe6d9ff4f572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:57.896292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d2865-d9dc-45ea-97b4-0ec394740e84 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.896292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d2865-d9dc-45ea-97b4-0ec394740e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.896462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfacccf-530b-4f1a-a712-fe6d9ff4f572 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:57.908278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 759/1549] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:50:57.910445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aee65-cc25-4c96-8840-62c21bc826ea None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.910839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aee65-cc25-4c96-8840-62c21bc826ea None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.911209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aee65-cc25-4c96-8840-62c21bc826ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.915037 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-693d2865-d9dc-45ea-97b4-0ec394740e84 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:57.916515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-693d2865-d9dc-45ea-97b4-0ec394740e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:57.925596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 791/1550] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:57.927050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fab7d-473c-46b1-a2c0-2126ac444a29 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.927328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fab7d-473c-46b1-a2c0-2126ac444a29 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.927691 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fab7d-473c-46b1-a2c0-2126ac444a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.932303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-730aee65-cc25-4c96-8840-62c21bc826ea None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:57.933801 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-730aee65-cc25-4c96-8840-62c21bc826ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:57.942814 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 760/1551] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:57.944064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b80ce-3f35-4851-917b-8f5d83ac0dc8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.944348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b80ce-3f35-4851-917b-8f5d83ac0dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.944619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b80ce-3f35-4851-917b-8f5d83ac0dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.947822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2b3fab7d-473c-46b1-a2c0-2126ac444a29 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:57.949046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2b3fab7d-473c-46b1-a2c0-2126ac444a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:57.958065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 792/1552] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:57.959318 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44375fd2-6765-4473-8dc1-7b0a29cce8c8 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.959595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44375fd2-6765-4473-8dc1-7b0a29cce8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.959864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44375fd2-6765-4473-8dc1-7b0a29cce8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.965827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-957b80ce-3f35-4851-917b-8f5d83ac0dc8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:57.967191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-957b80ce-3f35-4851-917b-8f5d83ac0dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:57.975978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 761/1553] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:57.977125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd694318-0d29-4ab1-8707-a6b22f7556f0 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:57.977406 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd694318-0d29-4ab1-8707-a6b22f7556f0 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:57.977674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd694318-0d29-4ab1-8707-a6b22f7556f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:57.979959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-44375fd2-6765-4473-8dc1-7b0a29cce8c8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:57.981227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-44375fd2-6765-4473-8dc1-7b0a29cce8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:57.989790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 793/1554] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:57.990753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44375fd2-6765-4473-8dc1-7b0a29cce8c8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.001958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-cd694318-0d29-4ab1-8707-a6b22f7556f0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:58.002247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44375fd2-6765-4473-8dc1-7b0a29cce8c8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.003278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-cd694318-0d29-4ab1-8707-a6b22f7556f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:58.007880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5bad4-982d-4d84-8047-b0734d4e1c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.008566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5bad4-982d-4d84-8047-b0734d4e1c75 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.008849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5bad4-982d-4d84-8047-b0734d4e1c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.009110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5bad4-982d-4d84-8047-b0734d4e1c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.009752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5bad4-982d-4d84-8047-b0734d4e1c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.011408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5bad4-982d-4d84-8047-b0734d4e1c75 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.011607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 762/1555] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:58.012471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd694318-0d29-4ab1-8707-a6b22f7556f0 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.023150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd694318-0d29-4ab1-8707-a6b22f7556f0 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.025086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 794/1556] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:58.026525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb50c3-e928-4876-9656-c1c9d066cc64 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.026842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb50c3-e928-4876-9656-c1c9d066cc64 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.027152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb50c3-e928-4876-9656-c1c9d066cc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.028219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c895b-243d-456a-886a-d1a86e48f404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.028744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c895b-243d-456a-886a-d1a86e48f404 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.028966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c895b-243d-456a-886a-d1a86e48f404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.029234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c895b-243d-456a-886a-d1a86e48f404 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.029857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c895b-243d-456a-886a-d1a86e48f404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.030952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: WARNING py.warnings [None req-4c7c895b-243d-456a-886a-d1a86e48f404 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:50:58.030952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: warnings.warn(msg) Aug 29 23:50:58.030952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: Aug 29 23:50:58.032111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c895b-243d-456a-886a-d1a86e48f404 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.047876 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-62cb50c3-e928-4876-9656-c1c9d066cc64 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:58.049719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-62cb50c3-e928-4876-9656-c1c9d066cc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:58.057750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 795/1557] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:58.061527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f102ca-c805-4ddf-9322-8a300e217188 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.061527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f102ca-c805-4ddf-9322-8a300e217188 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.061527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f102ca-c805-4ddf-9322-8a300e217188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.079486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-73f102ca-c805-4ddf-9322-8a300e217188 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:58.080948 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-73f102ca-c805-4ddf-9322-8a300e217188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:58.088909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 796/1558] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:58.089684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f102ca-c805-4ddf-9322-8a300e217188 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.099682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f102ca-c805-4ddf-9322-8a300e217188 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.104259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff8ea5-72e4-4562-a9b3-2a5c385bd30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.104832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff8ea5-72e4-4562-a9b3-2a5c385bd30f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.105111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff8ea5-72e4-4562-a9b3-2a5c385bd30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.105381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff8ea5-72e4-4562-a9b3-2a5c385bd30f admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.106012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff8ea5-72e4-4562-a9b3-2a5c385bd30f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:50:58.106309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff8ea5-72e4-4562-a9b3-2a5c385bd30f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.107356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-baff8ea5-72e4-4562-a9b3-2a5c385bd30f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:50:58.107356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 29 23:50:58.107356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: Aug 29 23:50:58.108227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff8ea5-72e4-4562-a9b3-2a5c385bd30f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.120523 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 797/1559] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:50:58.124745 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff8ea5-72e4-4562-a9b3-2a5c385bd30f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.131158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff8ea5-72e4-4562-a9b3-2a5c385bd30f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.135864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410f522-eb59-4fe1-922b-746dcf56f420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.136438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410f522-eb59-4fe1-922b-746dcf56f420 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.136717 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410f522-eb59-4fe1-922b-746dcf56f420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.136988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410f522-eb59-4fe1-922b-746dcf56f420 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.137617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410f522-eb59-4fe1-922b-746dcf56f420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:50:58.137917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410f522-eb59-4fe1-922b-746dcf56f420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.139565 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410f522-eb59-4fe1-922b-746dcf56f420 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.152235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 798/1560] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:50:58.153480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ab4af-a406-4691-97f7-4732820434d6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.153948 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ab4af-a406-4691-97f7-4732820434d6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.154057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ab4af-a406-4691-97f7-4732820434d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.164210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 763/1561] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:58.165816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8201c-1ca9-4060-a367-96656d44dec5 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.166096 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8201c-1ca9-4060-a367-96656d44dec5 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.166363 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8201c-1ca9-4060-a367-96656d44dec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.173403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-934ab4af-a406-4691-97f7-4732820434d6 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:58.175699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-934ab4af-a406-4691-97f7-4732820434d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:58.186181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d7e8201c-1ca9-4060-a367-96656d44dec5 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:58.187530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d7e8201c-1ca9-4060-a367-96656d44dec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:58.218722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:50:58.245256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 799/1562] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:58.245256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ab4af-a406-4691-97f7-4732820434d6 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.248699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 764/1563] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:58.249579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8201c-1ca9-4060-a367-96656d44dec5 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.257204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ab4af-a406-4691-97f7-4732820434d6 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.259194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8201c-1ca9-4060-a367-96656d44dec5 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.262599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97962c2d-a17d-4c24-a1cf-3e726db0e338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.263201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97962c2d-a17d-4c24-a1cf-3e726db0e338 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.268717 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97962c2d-a17d-4c24-a1cf-3e726db0e338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.268717 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97962c2d-a17d-4c24-a1cf-3e726db0e338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.268947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2372d-d90a-4b75-a527-fdd278c6dc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.268947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2372d-d90a-4b75-a527-fdd278c6dc86 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.269151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97962c2d-a17d-4c24-a1cf-3e726db0e338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.269294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2372d-d90a-4b75-a527-fdd278c6dc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.269294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2372d-d90a-4b75-a527-fdd278c6dc86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.269294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2372d-d90a-4b75-a527-fdd278c6dc86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:50:58.269294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2372d-d90a-4b75-a527-fdd278c6dc86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.269661 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97962c2d-a17d-4c24-a1cf-3e726db0e338 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.269763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2372d-d90a-4b75-a527-fdd278c6dc86 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.270501 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:50:58.277382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 800/1564] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:58.278394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97962c2d-a17d-4c24-a1cf-3e726db0e338 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.280007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 765/1565] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:50:58.280944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2372d-d90a-4b75-a527-fdd278c6dc86 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.290606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97962c2d-a17d-4c24-a1cf-3e726db0e338 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.296740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2372d-d90a-4b75-a527-fdd278c6dc86 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.296740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb87f6-bbf4-4469-ba37-a53862580e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.296982 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7011146-1aa5-4c95-9c41-3ac7cf2b8a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.297102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb87f6-bbf4-4469-ba37-a53862580e1f admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.297206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7011146-1aa5-4c95-9c41-3ac7cf2b8a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.297305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb87f6-bbf4-4469-ba37-a53862580e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.297423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7011146-1aa5-4c95-9c41-3ac7cf2b8a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.297526 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb87f6-bbf4-4469-ba37-a53862580e1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.297633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7011146-1aa5-4c95-9c41-3ac7cf2b8a1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.297731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb87f6-bbf4-4469-ba37-a53862580e1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.297827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7011146-1aa5-4c95-9c41-3ac7cf2b8a1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:50:58.297827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7011146-1aa5-4c95-9c41-3ac7cf2b8a1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.298954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb87f6-bbf4-4469-ba37-a53862580e1f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.299709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7011146-1aa5-4c95-9c41-3ac7cf2b8a1d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.316316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 801/1566] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:50:58.317008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7011146-1aa5-4c95-9c41-3ac7cf2b8a1d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.329233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7011146-1aa5-4c95-9c41-3ac7cf2b8a1d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.336351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304a674-bb3e-43e3-ac8a-a9728ff660c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.337026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304a674-bb3e-43e3-ac8a-a9728ff660c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.337441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304a674-bb3e-43e3-ac8a-a9728ff660c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.337848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304a674-bb3e-43e3-ac8a-a9728ff660c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.338853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304a674-bb3e-43e3-ac8a-a9728ff660c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.341923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304a674-bb3e-43e3-ac8a-a9728ff660c0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.403948 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 766/1567] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/projects => generated 403 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:58.403948 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb87f6-bbf4-4469-ba37-a53862580e1f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.452577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 802/1568] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/projects => generated 389 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:58.453680 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304a674-bb3e-43e3-ac8a-a9728ff660c0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.467036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb87f6-bbf4-4469-ba37-a53862580e1f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.517632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304a674-bb3e-43e3-ac8a-a9728ff660c0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.522506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36827f1-8cbb-4958-9429-8f62589d2b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.523153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36827f1-8cbb-4958-9429-8f62589d2b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.523472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36827f1-8cbb-4958-9429-8f62589d2b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.523747 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36827f1-8cbb-4958-9429-8f62589d2b4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.524393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36827f1-8cbb-4958-9429-8f62589d2b4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:50:58.524736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36827f1-8cbb-4958-9429-8f62589d2b4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.525745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c888be-33fd-48b4-a41b-b1166f9ce3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.526348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c888be-33fd-48b4-a41b-b1166f9ce3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.526498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36827f1-8cbb-4958-9429-8f62589d2b4c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.526658 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c888be-33fd-48b4-a41b-b1166f9ce3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.526930 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c888be-33fd-48b4-a41b-b1166f9ce3d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.527566 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c888be-33fd-48b4-a41b-b1166f9ce3d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.529510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c888be-33fd-48b4-a41b-b1166f9ce3d6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.546028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 803/1569] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:50:58.546028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36827f1-8cbb-4958-9429-8f62589d2b4c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.556462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36827f1-8cbb-4958-9429-8f62589d2b4c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.561062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb562f-ed25-4426-9652-a54a85d6c855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.563732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb562f-ed25-4426-9652-a54a85d6c855 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.563732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb562f-ed25-4426-9652-a54a85d6c855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.563732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb562f-ed25-4426-9652-a54a85d6c855 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.563732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb562f-ed25-4426-9652-a54a85d6c855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:50:58.563732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb562f-ed25-4426-9652-a54a85d6c855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.565463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb562f-ed25-4426-9652-a54a85d6c855 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.577690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 804/1570] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:50:58.578550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb562f-ed25-4426-9652-a54a85d6c855 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.588232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb562f-ed25-4426-9652-a54a85d6c855 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.592748 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127a00b-73d9-412e-bd4c-cb97624a4fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.593382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127a00b-73d9-412e-bd4c-cb97624a4fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.604074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127a00b-73d9-412e-bd4c-cb97624a4fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.604074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127a00b-73d9-412e-bd4c-cb97624a4fd3 admin admin] PATH_INFO: `/v3/projects/fc38f27d21dc494d9039037c3286cf6e/users/a102531e70e243469ce2b84e71b2a9b9/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.608404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127a00b-73d9-412e-bd4c-cb97624a4fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc38f27d21dc494d9039037c3286cf6e, user_id=a102531e70e243469ce2b84e71b2a9b9, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.611199 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127a00b-73d9-412e-bd4c-cb97624a4fd3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.628890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 767/1571] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:58.632762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c888be-33fd-48b4-a41b-b1166f9ce3d6 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.639740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c888be-33fd-48b4-a41b-b1166f9ce3d6 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.644824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6afc86-5c69-4b16-aefd-530f068d266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.645421 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6afc86-5c69-4b16-aefd-530f068d266c admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.645695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6afc86-5c69-4b16-aefd-530f068d266c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.645964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6afc86-5c69-4b16-aefd-530f068d266c admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.646638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6afc86-5c69-4b16-aefd-530f068d266c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.648527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6afc86-5c69-4b16-aefd-530f068d266c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.685837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 805/1572] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:50:58 2022] PUT /identity/v3/projects/fc38f27d21dc494d9039037c3286cf6e/users/a102531e70e243469ce2b84e71b2a9b9/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:50:58.690845 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127a00b-73d9-412e-bd4c-cb97624a4fd3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.721937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 768/1573] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:58.722938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6afc86-5c69-4b16-aefd-530f068d266c admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.743662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:50:58.757081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127a00b-73d9-412e-bd4c-cb97624a4fd3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.777745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:50:58.795298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6afc86-5c69-4b16-aefd-530f068d266c admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.800822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de5f4a-b161-4f32-9151-a6f9af960dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.801436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de5f4a-b161-4f32-9151-a6f9af960dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.801718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de5f4a-b161-4f32-9151-a6f9af960dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.801993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de5f4a-b161-4f32-9151-a6f9af960dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.802678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de5f4a-b161-4f32-9151-a6f9af960dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.804630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de5f4a-b161-4f32-9151-a6f9af960dc9 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.821215 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aec66a-a1ad-4310-a1f9-1f97945082f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.821825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aec66a-a1ad-4310-a1f9-1f97945082f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.822103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aec66a-a1ad-4310-a1f9-1f97945082f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.822376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aec66a-a1ad-4310-a1f9-1f97945082f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.823042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aec66a-a1ad-4310-a1f9-1f97945082f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.824966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aec66a-a1ad-4310-a1f9-1f97945082f1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.890969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 769/1574] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/users => generated 302 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:58.891790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de5f4a-b161-4f32-9151-a6f9af960dc9 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.903118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de5f4a-b161-4f32-9151-a6f9af960dc9 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.903918 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 806/1575] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/projects => generated 391 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:58.904739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aec66a-a1ad-4310-a1f9-1f97945082f1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:58.966081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9061192-dc16-4307-996c-473652de81fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.966741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9061192-dc16-4307-996c-473652de81fc admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.967028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9061192-dc16-4307-996c-473652de81fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.967297 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9061192-dc16-4307-996c-473652de81fc admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.967832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aec66a-a1ad-4310-a1f9-1f97945082f1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:58.967954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9061192-dc16-4307-996c-473652de81fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.969829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9061192-dc16-4307-996c-473652de81fc admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:58.973288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eea6c5-eded-4b57-831f-bbc0dcb9dff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:58.973890 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea6c5-eded-4b57-831f-bbc0dcb9dff5 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:58.974165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea6c5-eded-4b57-831f-bbc0dcb9dff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:58.974442 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea6c5-eded-4b57-831f-bbc0dcb9dff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:58.975090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eea6c5-eded-4b57-831f-bbc0dcb9dff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:58.976992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eea6c5-eded-4b57-831f-bbc0dcb9dff5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.060563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 770/1576] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:59.060563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9061192-dc16-4307-996c-473652de81fc admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.069574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9061192-dc16-4307-996c-473652de81fc admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.074507 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151b1ad-d441-4e64-8b19-116eac63bd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.075121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151b1ad-d441-4e64-8b19-116eac63bd2a admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.075458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151b1ad-d441-4e64-8b19-116eac63bd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.075697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151b1ad-d441-4e64-8b19-116eac63bd2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.076333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151b1ad-d441-4e64-8b19-116eac63bd2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.078329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151b1ad-d441-4e64-8b19-116eac63bd2a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.087073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 807/1577] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:50:58 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:59.088003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eea6c5-eded-4b57-831f-bbc0dcb9dff5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.154316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eea6c5-eded-4b57-831f-bbc0dcb9dff5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.159222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2ccda-03a4-4e3d-ab01-46120556aa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.159916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2ccda-03a4-4e3d-ab01-46120556aa90 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.160193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2ccda-03a4-4e3d-ab01-46120556aa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.160470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2ccda-03a4-4e3d-ab01-46120556aa90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.161111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2ccda-03a4-4e3d-ab01-46120556aa90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.162763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2ccda-03a4-4e3d-ab01-46120556aa90 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.167086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 771/1578] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:50:59 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:59.168492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627dfaa-f7a9-47e1-bb2e-a9ff228be17a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.168824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627dfaa-f7a9-47e1-bb2e-a9ff228be17a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.169152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627dfaa-f7a9-47e1-bb2e-a9ff228be17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.173472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 808/1579] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:59.174782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2ccda-03a4-4e3d-ab01-46120556aa90 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.184584 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2ccda-03a4-4e3d-ab01-46120556aa90 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.188281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:50:59.201257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-5627dfaa-f7a9-47e1-bb2e-a9ff228be17a None None] MFA Rules not processed for user `a102531e70e243469ce2b84e71b2a9b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:50:59.202517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5627dfaa-f7a9-47e1-bb2e-a9ff228be17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:50:59.245987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bf72c-2ce6-4422-b96a-44f51c010e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.246630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bf72c-2ce6-4422-b96a-44f51c010e44 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.246913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bf72c-2ce6-4422-b96a-44f51c010e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.247190 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bf72c-2ce6-4422-b96a-44f51c010e44 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.247824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bf72c-2ce6-4422-b96a-44f51c010e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.249788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bf72c-2ce6-4422-b96a-44f51c010e44 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.270233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 772/1580] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:50:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:50:59.271322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627dfaa-f7a9-47e1-bb2e-a9ff228be17a None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.281347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627dfaa-f7a9-47e1-bb2e-a9ff228be17a None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.286200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49291ed7-6e95-47c2-baea-5d3cd37ae497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.286816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49291ed7-6e95-47c2-baea-5d3cd37ae497 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.287091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49291ed7-6e95-47c2-baea-5d3cd37ae497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.287357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49291ed7-6e95-47c2-baea-5d3cd37ae497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.287990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49291ed7-6e95-47c2-baea-5d3cd37ae497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.289580 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49291ed7-6e95-47c2-baea-5d3cd37ae497 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.300589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 773/1581] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:59.301731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49291ed7-6e95-47c2-baea-5d3cd37ae497 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.311585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49291ed7-6e95-47c2-baea-5d3cd37ae497 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.317627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ae42c-338a-4518-9755-819d793d86ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.318395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ae42c-338a-4518-9755-819d793d86ed admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.318700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ae42c-338a-4518-9755-819d793d86ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.318970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ae42c-338a-4518-9755-819d793d86ed admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.319592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ae42c-338a-4518-9755-819d793d86ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.321521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ae42c-338a-4518-9755-819d793d86ed admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.360033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 809/1582] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:50:59 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:59.360880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bf72c-2ce6-4422-b96a-44f51c010e44 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.371321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bf72c-2ce6-4422-b96a-44f51c010e44 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.375930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5f403-2cc8-42b3-a17d-8a741c154f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.376521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5f403-2cc8-42b3-a17d-8a741c154f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.376800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5f403-2cc8-42b3-a17d-8a741c154f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.377071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5f403-2cc8-42b3-a17d-8a741c154f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.378815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5f403-2cc8-42b3-a17d-8a741c154f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.380423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5f403-2cc8-42b3-a17d-8a741c154f28 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.390861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 810/1583] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:59.391654 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5f403-2cc8-42b3-a17d-8a741c154f28 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.401737 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5f403-2cc8-42b3-a17d-8a741c154f28 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.406355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d2e7d-be42-41aa-8b29-0cec8028d028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.406905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d2e7d-be42-41aa-8b29-0cec8028d028 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.407185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d2e7d-be42-41aa-8b29-0cec8028d028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.407461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d2e7d-be42-41aa-8b29-0cec8028d028 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.408067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d2e7d-be42-41aa-8b29-0cec8028d028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.410577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d2e7d-be42-41aa-8b29-0cec8028d028 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.439935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 774/1584] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:50:59 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:59.440877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ae42c-338a-4518-9755-819d793d86ed admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.447063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:50:59.450882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ae42c-338a-4518-9755-819d793d86ed admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.455585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f297671-48ad-40a2-b365-792e799e0143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.456167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f297671-48ad-40a2-b365-792e799e0143 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.456440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f297671-48ad-40a2-b365-792e799e0143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.456777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f297671-48ad-40a2-b365-792e799e0143 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.457401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f297671-48ad-40a2-b365-792e799e0143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.459464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f297671-48ad-40a2-b365-792e799e0143 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.520046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 811/1585] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:50:59 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:59.520968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d2e7d-be42-41aa-8b29-0cec8028d028 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.530443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d2e7d-be42-41aa-8b29-0cec8028d028 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.535657 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1c23-0fd3-4ae8-98a0-75381fbd6015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.536233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1c23-0fd3-4ae8-98a0-75381fbd6015 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.536510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1c23-0fd3-4ae8-98a0-75381fbd6015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.536777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1c23-0fd3-4ae8-98a0-75381fbd6015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.537404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c1c23-0fd3-4ae8-98a0-75381fbd6015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.539023 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c1c23-0fd3-4ae8-98a0-75381fbd6015 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.549009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 775/1586] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:50:59 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:59.551377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 812/1587] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:59.558861 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f297671-48ad-40a2-b365-792e799e0143 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.558996 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1c23-0fd3-4ae8-98a0-75381fbd6015 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.560755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f297671-48ad-40a2-b365-792e799e0143 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.561396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1c23-0fd3-4ae8-98a0-75381fbd6015 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.566947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae594a-e882-4749-bc60-f2e9837a45c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.567128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524629cf-ebd0-402e-b1a0-56914f96c33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.567545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae594a-e882-4749-bc60-f2e9837a45c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.567729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524629cf-ebd0-402e-b1a0-56914f96c33b admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.567827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae594a-e882-4749-bc60-f2e9837a45c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.568009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524629cf-ebd0-402e-b1a0-56914f96c33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.568153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae594a-e882-4749-bc60-f2e9837a45c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.568361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524629cf-ebd0-402e-b1a0-56914f96c33b admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.568696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae594a-e882-4749-bc60-f2e9837a45c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.568895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524629cf-ebd0-402e-b1a0-56914f96c33b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.570470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae594a-e882-4749-bc60-f2e9837a45c6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.570841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524629cf-ebd0-402e-b1a0-56914f96c33b admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.583300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 813/1588] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:59.584080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae594a-e882-4749-bc60-f2e9837a45c6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.593773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae594a-e882-4749-bc60-f2e9837a45c6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.599648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70f8ea-2c2b-4578-a663-006200480b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.600237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70f8ea-2c2b-4578-a663-006200480b2f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.600511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70f8ea-2c2b-4578-a663-006200480b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.600779 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70f8ea-2c2b-4578-a663-006200480b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.601405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70f8ea-2c2b-4578-a663-006200480b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.603041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70f8ea-2c2b-4578-a663-006200480b2f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.613741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 814/1589] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:59.614552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70f8ea-2c2b-4578-a663-006200480b2f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.624158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70f8ea-2c2b-4578-a663-006200480b2f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.629584 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ca3d3-00f6-444c-ab72-b404c38f7bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.630159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ca3d3-00f6-444c-ab72-b404c38f7bea admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.630452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ca3d3-00f6-444c-ab72-b404c38f7bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.630732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ca3d3-00f6-444c-ab72-b404c38f7bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.631350 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ca3d3-00f6-444c-ab72-b404c38f7bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.632949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ca3d3-00f6-444c-ab72-b404c38f7bea admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.642716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 815/1590] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:59.643618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ca3d3-00f6-444c-ab72-b404c38f7bea admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.653676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ca3d3-00f6-444c-ab72-b404c38f7bea admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.658328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488f4c7-b799-425e-b1ad-9d0ecc863ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.659038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488f4c7-b799-425e-b1ad-9d0ecc863ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.659341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488f4c7-b799-425e-b1ad-9d0ecc863ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.659578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488f4c7-b799-425e-b1ad-9d0ecc863ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.660211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488f4c7-b799-425e-b1ad-9d0ecc863ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.661801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488f4c7-b799-425e-b1ad-9d0ecc863ba4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.672520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 816/1591] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:59.673280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488f4c7-b799-425e-b1ad-9d0ecc863ba4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.683211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488f4c7-b799-425e-b1ad-9d0ecc863ba4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.687769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342ae3f-73aa-43a7-8c01-2efc0b2504e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.688342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342ae3f-73aa-43a7-8c01-2efc0b2504e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.688622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342ae3f-73aa-43a7-8c01-2efc0b2504e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.688888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342ae3f-73aa-43a7-8c01-2efc0b2504e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.689505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342ae3f-73aa-43a7-8c01-2efc0b2504e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.692072 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342ae3f-73aa-43a7-8c01-2efc0b2504e9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.702001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 817/1592] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:59.702807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342ae3f-73aa-43a7-8c01-2efc0b2504e9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.712911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342ae3f-73aa-43a7-8c01-2efc0b2504e9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.715930 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 776/1593] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:50:59 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:50:59.716697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524629cf-ebd0-402e-b1a0-56914f96c33b admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.719645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c6156-35c7-4f13-ba00-40c9e5418eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.720262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c6156-35c7-4f13-ba00-40c9e5418eac admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.720537 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c6156-35c7-4f13-ba00-40c9e5418eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.720808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c6156-35c7-4f13-ba00-40c9e5418eac admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.721890 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c6156-35c7-4f13-ba00-40c9e5418eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.723801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c6156-35c7-4f13-ba00-40c9e5418eac admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.734470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 818/1594] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:50:59.735299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c6156-35c7-4f13-ba00-40c9e5418eac admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.745717 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c6156-35c7-4f13-ba00-40c9e5418eac admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.784687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524629cf-ebd0-402e-b1a0-56914f96c33b admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:50:59.789375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d9623-6f72-4b8d-81f9-69edae4c709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.790033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d9623-6f72-4b8d-81f9-69edae4c709a admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.790320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d9623-6f72-4b8d-81f9-69edae4c709a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.790629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d9623-6f72-4b8d-81f9-69edae4c709a admin admin] PATH_INFO: `/v3/projects/a8ada08f998844e997babd510083c205/users/2b88462a0d8d4e3fb4672a3f1d219c28/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.794357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:50:59.804578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850cacf-3470-464e-9d2b-abdbb82529c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:50:59.807830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850cacf-3470-464e-9d2b-abdbb82529c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:50:59.807830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850cacf-3470-464e-9d2b-abdbb82529c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:50:59.807830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850cacf-3470-464e-9d2b-abdbb82529c9 admin admin] PATH_INFO: `/v3/projects/49634e29b4934624a8b94d5385098c12/users/8402373cf55e47fbb965123895642d70/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:50:59.808117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d9623-6f72-4b8d-81f9-69edae4c709a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ada08f998844e997babd510083c205, user_id=2b88462a0d8d4e3fb4672a3f1d219c28, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.808117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: WARNING py.warnings [None req-3a6d9623-6f72-4b8d-81f9-69edae4c709a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:50:59.808117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: warnings.warn(msg) Aug 29 23:50:59.808117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: Aug 29 23:50:59.809666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d9623-6f72-4b8d-81f9-69edae4c709a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.821330 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850cacf-3470-464e-9d2b-abdbb82529c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49634e29b4934624a8b94d5385098c12, user_id=8402373cf55e47fbb965123895642d70, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:50:59.824063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850cacf-3470-464e-9d2b-abdbb82529c9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:50:59.914514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 777/1595] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:50:59 2022] PUT /identity/v3/projects/a8ada08f998844e997babd510083c205/users/2b88462a0d8d4e3fb4672a3f1d219c28/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:50:59.915304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d9623-6f72-4b8d-81f9-69edae4c709a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.949041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 819/1596] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:50:59 2022] PUT /identity/v3/projects/49634e29b4934624a8b94d5385098c12/users/8402373cf55e47fbb965123895642d70/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:50:59.949909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850cacf-3470-464e-9d2b-abdbb82529c9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:50:59.978570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d9623-6f72-4b8d-81f9-69edae4c709a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.012818 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850cacf-3470-464e-9d2b-abdbb82529c9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.017606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7ca58-441f-4b2f-97d4-306de271a0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.018213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7ca58-441f-4b2f-97d4-306de271a0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.018532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7ca58-441f-4b2f-97d4-306de271a0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.018804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7ca58-441f-4b2f-97d4-306de271a0f3 admin admin] PATH_INFO: `/v3/projects/3fd4107165644976b1d8cdbeec3c8711/users/90b2c6a74c29443b94d1577b39c7ce0b/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.031607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:00.033420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7ca58-441f-4b2f-97d4-306de271a0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd4107165644976b1d8cdbeec3c8711, user_id=90b2c6a74c29443b94d1577b39c7ce0b, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.036159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7ca58-441f-4b2f-97d4-306de271a0f3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.038812 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ceed8b-a2ff-467e-8f52-fcbdaba99aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.039422 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceed8b-a2ff-467e-8f52-fcbdaba99aec admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.039700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceed8b-a2ff-467e-8f52-fcbdaba99aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.039972 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceed8b-a2ff-467e-8f52-fcbdaba99aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.040613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ceed8b-a2ff-467e-8f52-fcbdaba99aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.042234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ceed8b-a2ff-467e-8f52-fcbdaba99aec admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.052844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 778/1597] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:00.053666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ceed8b-a2ff-467e-8f52-fcbdaba99aec admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.063054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ceed8b-a2ff-467e-8f52-fcbdaba99aec admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.072327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3786db-9c19-41f3-afbe-4c4bad13155f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.072327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3786db-9c19-41f3-afbe-4c4bad13155f admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.072327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3786db-9c19-41f3-afbe-4c4bad13155f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.072327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3786db-9c19-41f3-afbe-4c4bad13155f admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.073514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3786db-9c19-41f3-afbe-4c4bad13155f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.073514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3786db-9c19-41f3-afbe-4c4bad13155f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.082348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 779/1598] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:00.083195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3786db-9c19-41f3-afbe-4c4bad13155f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.092681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3786db-9c19-41f3-afbe-4c4bad13155f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.097575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ed991-2f01-44e1-a9df-401f2d5b356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.098155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ed991-2f01-44e1-a9df-401f2d5b356d admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.098450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ed991-2f01-44e1-a9df-401f2d5b356d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.098729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ed991-2f01-44e1-a9df-401f2d5b356d admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.099367 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ed991-2f01-44e1-a9df-401f2d5b356d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.100940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ed991-2f01-44e1-a9df-401f2d5b356d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.111262 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 780/1599] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:00.112687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ed991-2f01-44e1-a9df-401f2d5b356d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.122057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ed991-2f01-44e1-a9df-401f2d5b356d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.126643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e94cc8-851c-4237-96a0-b72d0ae81b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.127294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e94cc8-851c-4237-96a0-b72d0ae81b88 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.128192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e94cc8-851c-4237-96a0-b72d0ae81b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.128506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e94cc8-851c-4237-96a0-b72d0ae81b88 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.129148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e94cc8-851c-4237-96a0-b72d0ae81b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.131048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e94cc8-851c-4237-96a0-b72d0ae81b88 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.146241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 820/1600] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:50:59 2022] PUT /identity/v3/projects/3fd4107165644976b1d8cdbeec3c8711/users/90b2c6a74c29443b94d1577b39c7ce0b/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:00.147381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7ca58-441f-4b2f-97d4-306de271a0f3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.205708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 781/1601] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:00 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:00.207281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff78b3-01a3-48dc-b9cd-4c1cbf52b4cc None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.207506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff78b3-01a3-48dc-b9cd-4c1cbf52b4cc None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.207773 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff78b3-01a3-48dc-b9cd-4c1cbf52b4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.209235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7ca58-441f-4b2f-97d4-306de271a0f3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.213930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccc325-6691-4037-8dc4-b874ce334a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.215888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccc325-6691-4037-8dc4-b874ce334a64 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.215888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccc325-6691-4037-8dc4-b874ce334a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.215888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccc325-6691-4037-8dc4-b874ce334a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.215888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccc325-6691-4037-8dc4-b874ce334a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.217403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccc325-6691-4037-8dc4-b874ce334a64 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.227928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-66ff78b3-01a3-48dc-b9cd-4c1cbf52b4cc None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:00.229125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-66ff78b3-01a3-48dc-b9cd-4c1cbf52b4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:00.229238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 821/1602] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:00.230739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becea86-1a39-475d-83f0-19bf0e299bde None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.231137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becea86-1a39-475d-83f0-19bf0e299bde None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.231417 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becea86-1a39-475d-83f0-19bf0e299bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.238841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 782/1603] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:00.241137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff78b3-01a3-48dc-b9cd-4c1cbf52b4cc None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.249396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff78b3-01a3-48dc-b9cd-4c1cbf52b4cc None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.254268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e8f09-eebf-4e5b-8e65-72e77c7eedba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.255041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e8f09-eebf-4e5b-8e65-72e77c7eedba admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.255309 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e8f09-eebf-4e5b-8e65-72e77c7eedba admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.255575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e8f09-eebf-4e5b-8e65-72e77c7eedba admin admin] PATH_INFO: `/v3/projects/2d358f3073d446f49dc51777a7afc30e/users/db604b7af623452fac9b3712b6978168/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.263987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5becea86-1a39-475d-83f0-19bf0e299bde None None] MFA Rules not processed for user `8402373cf55e47fbb965123895642d70`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:00.265205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5becea86-1a39-475d-83f0-19bf0e299bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:00.271170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e8f09-eebf-4e5b-8e65-72e77c7eedba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d358f3073d446f49dc51777a7afc30e, user_id=db604b7af623452fac9b3712b6978168, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.273810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e8f09-eebf-4e5b-8e65-72e77c7eedba admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.306352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:00.325685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 783/1604] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:00 2022] PUT /identity/v3/projects/2d358f3073d446f49dc51777a7afc30e/users/db604b7af623452fac9b3712b6978168/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:00.326873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e8f09-eebf-4e5b-8e65-72e77c7eedba admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.333107 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 822/1605] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:00.333875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becea86-1a39-475d-83f0-19bf0e299bde None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.349651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becea86-1a39-475d-83f0-19bf0e299bde None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.398897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e8f09-eebf-4e5b-8e65-72e77c7eedba admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.403738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e48736-2bb0-4613-86cd-0ec33ac55cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.404397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e48736-2bb0-4613-86cd-0ec33ac55cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.404676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e48736-2bb0-4613-86cd-0ec33ac55cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.404950 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e48736-2bb0-4613-86cd-0ec33ac55cd5 admin admin] PATH_INFO: `/v3/projects/cb38bc3ae5c14e4c8385809d84573600/users/734d44438bfb4a79a68aa6705e49b1e9/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.407482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136053b-d9a6-4da7-893e-7286059e0cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.408144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136053b-d9a6-4da7-893e-7286059e0cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.408983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136053b-d9a6-4da7-893e-7286059e0cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.409268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136053b-d9a6-4da7-893e-7286059e0cdc admin admin] PATH_INFO: `/v3/projects/80d82515633f4c2faa34d3225a6f26a6/users/34ef7ee27c3948ac9df1ff94ce24618e/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.423367 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e48736-2bb0-4613-86cd-0ec33ac55cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb38bc3ae5c14e4c8385809d84573600, user_id=734d44438bfb4a79a68aa6705e49b1e9, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.425683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136053b-d9a6-4da7-893e-7286059e0cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d82515633f4c2faa34d3225a6f26a6, user_id=34ef7ee27c3948ac9df1ff94ce24618e, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.426813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e48736-2bb0-4613-86cd-0ec33ac55cd5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.428262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136053b-d9a6-4da7-893e-7286059e0cdc admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.492143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 784/1606] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:00 2022] PUT /identity/v3/projects/cb38bc3ae5c14e4c8385809d84573600/users/734d44438bfb4a79a68aa6705e49b1e9/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:00.493364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc6358-274b-4150-8dd1-37804f879a73 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.493651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc6358-274b-4150-8dd1-37804f879a73 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.493920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc6358-274b-4150-8dd1-37804f879a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.502578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 823/1607] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:00 2022] PUT /identity/v3/projects/80d82515633f4c2faa34d3225a6f26a6/users/34ef7ee27c3948ac9df1ff94ce24618e/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:00.503963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136053b-d9a6-4da7-893e-7286059e0cdc admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.513591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-15dc6358-274b-4150-8dd1-37804f879a73 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:00.514829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-15dc6358-274b-4150-8dd1-37804f879a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:00.569882 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136053b-d9a6-4da7-893e-7286059e0cdc admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.574431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ebe8a-81fc-49da-9d56-f1c3b77ecaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.575043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ebe8a-81fc-49da-9d56-f1c3b77ecaab admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.575319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ebe8a-81fc-49da-9d56-f1c3b77ecaab admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.575588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ebe8a-81fc-49da-9d56-f1c3b77ecaab admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.576224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ebe8a-81fc-49da-9d56-f1c3b77ecaab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.576534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 785/1608] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:00.577404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc6358-274b-4150-8dd1-37804f879a73 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.577832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ebe8a-81fc-49da-9d56-f1c3b77ecaab admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.585254 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:00.589225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc6358-274b-4150-8dd1-37804f879a73 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.589867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 824/1609] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:00.590677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ebe8a-81fc-49da-9d56-f1c3b77ecaab admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.594030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027347b-2a10-4774-b662-51e0e446f78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.594725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027347b-2a10-4774-b662-51e0e446f78f admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.595006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027347b-2a10-4774-b662-51e0e446f78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.595279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027347b-2a10-4774-b662-51e0e446f78f admin admin] PATH_INFO: `/v3/projects/f71b0d227a5147218377cab6f22a9b52/users/76ab113b2f3144da87bf572cc0b2344f/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.612145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027347b-2a10-4774-b662-51e0e446f78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71b0d227a5147218377cab6f22a9b52, user_id=76ab113b2f3144da87bf572cc0b2344f, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.614857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027347b-2a10-4774-b662-51e0e446f78f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.649373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ebe8a-81fc-49da-9d56-f1c3b77ecaab admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.653142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a5a33-8eef-49c0-b9e0-47f713f97998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.653796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5a33-8eef-49c0-b9e0-47f713f97998 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.654072 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5a33-8eef-49c0-b9e0-47f713f97998 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.654342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5a33-8eef-49c0-b9e0-47f713f97998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.664175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a5a33-8eef-49c0-b9e0-47f713f97998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.665605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a5a33-8eef-49c0-b9e0-47f713f97998 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.675325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 825/1610] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:00.676549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b4f4-c01a-4ddb-a48d-b1e8a69872f0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.676834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b4f4-c01a-4ddb-a48d-b1e8a69872f0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.677140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b4f4-c01a-4ddb-a48d-b1e8a69872f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.685127 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:00.685954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 786/1611] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:00 2022] PUT /identity/v3/projects/f71b0d227a5147218377cab6f22a9b52/users/76ab113b2f3144da87bf572cc0b2344f/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:00.686953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027347b-2a10-4774-b662-51e0e446f78f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.697445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-49d4b4f4-c01a-4ddb-a48d-b1e8a69872f0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:00.698949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-49d4b4f4-c01a-4ddb-a48d-b1e8a69872f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:00.761659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027347b-2a10-4774-b662-51e0e446f78f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.763043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 826/1612] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:00.764291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eeebb-29b8-43dc-b553-f4339c953f1d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.764575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eeebb-29b8-43dc-b553-f4339c953f1d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.764846 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eeebb-29b8-43dc-b553-f4339c953f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.772611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162ff6d-0e6d-4f1a-b111-2522406f2116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.773476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162ff6d-0e6d-4f1a-b111-2522406f2116 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.773754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162ff6d-0e6d-4f1a-b111-2522406f2116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.774027 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162ff6d-0e6d-4f1a-b111-2522406f2116 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.774705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162ff6d-0e6d-4f1a-b111-2522406f2116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.776686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162ff6d-0e6d-4f1a-b111-2522406f2116 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.786372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d59eeebb-29b8-43dc-b553-f4339c953f1d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:00.788197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d59eeebb-29b8-43dc-b553-f4339c953f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:00.799370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 827/1613] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:00.801596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af592328-442f-4d21-aca2-baf805fd38b8 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.802008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af592328-442f-4d21-aca2-baf805fd38b8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.802296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af592328-442f-4d21-aca2-baf805fd38b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.822675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-af592328-442f-4d21-aca2-baf805fd38b8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:00.823898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-af592328-442f-4d21-aca2-baf805fd38b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:00.835406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 828/1614] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:00.835406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af592328-442f-4d21-aca2-baf805fd38b8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.894529 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af592328-442f-4d21-aca2-baf805fd38b8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.898240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa5a09-8ff6-4f8d-a05a-16efaa9dbe57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.898877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa5a09-8ff6-4f8d-a05a-16efaa9dbe57 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.899152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa5a09-8ff6-4f8d-a05a-16efaa9dbe57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.899471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa5a09-8ff6-4f8d-a05a-16efaa9dbe57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.908732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa5a09-8ff6-4f8d-a05a-16efaa9dbe57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:00.910772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa5a09-8ff6-4f8d-a05a-16efaa9dbe57 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:00.920240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 829/1615] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:00.925900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa5a09-8ff6-4f8d-a05a-16efaa9dbe57 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:00.926204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 787/1616] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:00 2022] POST /identity/v3/projects => generated 403 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:00.926204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9384d4-241d-4a8e-bc9b-6aaae6a22e84 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:00.926204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9384d4-241d-4a8e-bc9b-6aaae6a22e84 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:00.926204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9384d4-241d-4a8e-bc9b-6aaae6a22e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:00.934263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa5a09-8ff6-4f8d-a05a-16efaa9dbe57 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:00.956150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-4e9384d4-241d-4a8e-bc9b-6aaae6a22e84 None None] MFA Rules not processed for user `76ab113b2f3144da87bf572cc0b2344f`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:00.957376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4e9384d4-241d-4a8e-bc9b-6aaae6a22e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:00.999244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdff8c8-d0bc-4228-9038-07f712911094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:00.999928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdff8c8-d0bc-4228-9038-07f712911094 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.000200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdff8c8-d0bc-4228-9038-07f712911094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.000476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdff8c8-d0bc-4228-9038-07f712911094 admin admin] PATH_INFO: `/v3/projects/803ce2e918ff438fbb4aa74065c37cf8/users/05b38113df444c48898ace21dcea68db/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.032542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdff8c8-d0bc-4228-9038-07f712911094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=803ce2e918ff438fbb4aa74065c37cf8, user_id=05b38113df444c48898ace21dcea68db, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:01.036661 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdff8c8-d0bc-4228-9038-07f712911094 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:01.046771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 788/1617] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:01.047656 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9384d4-241d-4a8e-bc9b-6aaae6a22e84 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:01.108784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9384d4-241d-4a8e-bc9b-6aaae6a22e84 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:01.112960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a1c13-2cc6-4c54-97ca-1ac6ba3587cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:01.113593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a1c13-2cc6-4c54-97ca-1ac6ba3587cf service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.118395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a1c13-2cc6-4c54-97ca-1ac6ba3587cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.118395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a1c13-2cc6-4c54-97ca-1ac6ba3587cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.124101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a1c13-2cc6-4c54-97ca-1ac6ba3587cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:01.125548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a1c13-2cc6-4c54-97ca-1ac6ba3587cf service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:01.130646 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:01.142476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 789/1618] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:01.150681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a1c13-2cc6-4c54-97ca-1ac6ba3587cf service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:01.150681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-805a1c13-2cc6-4c54-97ca-1ac6ba3587cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:01.150911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 830/1619] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:00 2022] PUT /identity/v3/projects/803ce2e918ff438fbb4aa74065c37cf8/users/05b38113df444c48898ace21dcea68db/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:01.150911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdff8c8-d0bc-4228-9038-07f712911094 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:01.151257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7bdff8c8-d0bc-4228-9038-07f712911094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:01.159672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a1c13-2cc6-4c54-97ca-1ac6ba3587cf service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:01.176367 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:01.218351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdff8c8-d0bc-4228-9038-07f712911094 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:01.223201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54806c92-b8f7-47b5-943e-0b93c18cd8f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:01.223807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54806c92-b8f7-47b5-943e-0b93c18cd8f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.224088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54806c92-b8f7-47b5-943e-0b93c18cd8f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.224193 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37adcb7-f352-4680-ac95-445d3fe9714e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:01.224372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54806c92-b8f7-47b5-943e-0b93c18cd8f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.224960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37adcb7-f352-4680-ac95-445d3fe9714e service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.225407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37adcb7-f352-4680-ac95-445d3fe9714e service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.225757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37adcb7-f352-4680-ac95-445d3fe9714e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.288820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54806c92-b8f7-47b5-943e-0b93c18cd8f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:01.290193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54806c92-b8f7-47b5-943e-0b93c18cd8f6 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:01.294747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37adcb7-f352-4680-ac95-445d3fe9714e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:01.296201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37adcb7-f352-4680-ac95-445d3fe9714e service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:01.301687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 831/1620] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:01.302566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54806c92-b8f7-47b5-943e-0b93c18cd8f6 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:01.307966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 790/1621] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:01.309222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6ccf-7167-4c4a-a64f-bced67f31841 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.309506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6ccf-7167-4c4a-a64f-bced67f31841 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.309780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6ccf-7167-4c4a-a64f-bced67f31841 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.311627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 791/1622] 10.222.0.58 () {54 vars in 962 bytes} [Mon Aug 29 23:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:51:01.312246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d6ccf-7167-4c4a-a64f-bced67f31841 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:01.312872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54806c92-b8f7-47b5-943e-0b93c18cd8f6 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:01.319150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be3028-fd34-4959-8b19-7a5632a30982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:01.319864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be3028-fd34-4959-8b19-7a5632a30982 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.320213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be3028-fd34-4959-8b19-7a5632a30982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.320503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be3028-fd34-4959-8b19-7a5632a30982 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.321282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be3028-fd34-4959-8b19-7a5632a30982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:01.323483 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be3028-fd34-4959-8b19-7a5632a30982 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:01.331542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d6ccf-7167-4c4a-a64f-bced67f31841 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:01.336810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676e6b3-df53-47ea-8c43-f32240879b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:01.337424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676e6b3-df53-47ea-8c43-f32240879b14 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.337697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676e6b3-df53-47ea-8c43-f32240879b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.337980 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676e6b3-df53-47ea-8c43-f32240879b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.338655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676e6b3-df53-47ea-8c43-f32240879b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:01.340307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676e6b3-df53-47ea-8c43-f32240879b14 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:01.352988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 792/1623] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:01.356301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5c372-4e3c-460f-802f-fd74d9817698 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.356643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5c372-4e3c-460f-802f-fd74d9817698 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.356991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5c372-4e3c-460f-802f-fd74d9817698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.397419 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-43e5c372-4e3c-460f-802f-fd74d9817698 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:01.398630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-43e5c372-4e3c-460f-802f-fd74d9817698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:01.443142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 832/1624] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:01 2022] POST /identity/v3/users => generated 308 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:01.443142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be3028-fd34-4959-8b19-7a5632a30982 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:01.468451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be3028-fd34-4959-8b19-7a5632a30982 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:01.477655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc50d0b-6a64-4137-9a88-10cf8be3f675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:01.478609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc50d0b-6a64-4137-9a88-10cf8be3f675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.479094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc50d0b-6a64-4137-9a88-10cf8be3f675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.479715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc50d0b-6a64-4137-9a88-10cf8be3f675 admin admin] PATH_INFO: `/v3/projects/803ce2e918ff438fbb4aa74065c37cf8/users/05b38113df444c48898ace21dcea68db/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.483219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc50d0b-6a64-4137-9a88-10cf8be3f675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=803ce2e918ff438fbb4aa74065c37cf8, user_id=05b38113df444c48898ace21dcea68db, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:01.488307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc50d0b-6a64-4137-9a88-10cf8be3f675 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:01.517782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 833/1625] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:01 2022] PUT /identity/v3/projects/803ce2e918ff438fbb4aa74065c37cf8/users/05b38113df444c48898ace21dcea68db/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:01.519879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc50d0b-6a64-4137-9a88-10cf8be3f675 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:01.529851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 793/1626] 10.222.0.58 () {58 vars in 1059 bytes} [Mon Aug 29 23:51:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:01.534228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734f746-f9b6-4b98-b993-b5ea9c1a0a72 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.534966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734f746-f9b6-4b98-b993-b5ea9c1a0a72 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.535590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734f746-f9b6-4b98-b993-b5ea9c1a0a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.585621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-2734f746-f9b6-4b98-b993-b5ea9c1a0a72 None None] MFA Rules not processed for user `05b38113df444c48898ace21dcea68db`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:01.586907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2734f746-f9b6-4b98-b993-b5ea9c1a0a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:01.613745 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc50d0b-6a64-4137-9a88-10cf8be3f675 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:01.619000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c13b3-7133-4ac0-9dec-0fb0238c3b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:01.619962 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c13b3-7133-4ac0-9dec-0fb0238c3b40 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.620618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c13b3-7133-4ac0-9dec-0fb0238c3b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.620618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c13b3-7133-4ac0-9dec-0fb0238c3b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.621195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c13b3-7133-4ac0-9dec-0fb0238c3b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:01.622841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c13b3-7133-4ac0-9dec-0fb0238c3b40 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:01.634917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 834/1627] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:01.635812 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c13b3-7133-4ac0-9dec-0fb0238c3b40 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:01.706737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 794/1628] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:01.709865 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734f746-f9b6-4b98-b993-b5ea9c1a0a72 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:01.730737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734f746-f9b6-4b98-b993-b5ea9c1a0a72 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:01.741155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88875bf4-ebee-4372-b906-eb28854245a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:01.742124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c13b3-7133-4ac0-9dec-0fb0238c3b40 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:01.742286 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88875bf4-ebee-4372-b906-eb28854245a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.742885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88875bf4-ebee-4372-b906-eb28854245a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.743433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88875bf4-ebee-4372-b906-eb28854245a4 admin admin] PATH_INFO: `/v3/projects/134263c87f7e441c90e1059cf4027576/users/f8bf97b21005495cb1ca9aac9eef516a/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.750365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e0d9e-61e8-402d-8142-6b7b94eeb344 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:01.751174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e0d9e-61e8-402d-8142-6b7b94eeb344 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:01.751641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e0d9e-61e8-402d-8142-6b7b94eeb344 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:01.752051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e0d9e-61e8-402d-8142-6b7b94eeb344 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:01.767911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88875bf4-ebee-4372-b906-eb28854245a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=134263c87f7e441c90e1059cf4027576, user_id=f8bf97b21005495cb1ca9aac9eef516a, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:01.772974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88875bf4-ebee-4372-b906-eb28854245a4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:01.827308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e0d9e-61e8-402d-8142-6b7b94eeb344 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:01.829236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e0d9e-61e8-402d-8142-6b7b94eeb344 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:01.840335 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 835/1629] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:01.841480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e0d9e-61e8-402d-8142-6b7b94eeb344 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:01.881247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:01.894629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 795/1630] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:01 2022] PUT /identity/v3/projects/134263c87f7e441c90e1059cf4027576/users/f8bf97b21005495cb1ca9aac9eef516a/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:01.895728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88875bf4-ebee-4372-b906-eb28854245a4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:01.937991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e0d9e-61e8-402d-8142-6b7b94eeb344 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:01.977774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:02.051930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f050789-6537-4f93-a873-de732bb9394e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:02.052564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f050789-6537-4f93-a873-de732bb9394e service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:02.053919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f050789-6537-4f93-a873-de732bb9394e service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:02.054165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f050789-6537-4f93-a873-de732bb9394e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:02.100408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88875bf4-ebee-4372-b906-eb28854245a4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:02.115596 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845a718-d009-4fe9-954b-c2e424d21f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:02.117627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845a718-d009-4fe9-954b-c2e424d21f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:02.118199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845a718-d009-4fe9-954b-c2e424d21f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:02.118498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845a718-d009-4fe9-954b-c2e424d21f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:02.121124 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845a718-d009-4fe9-954b-c2e424d21f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:02.126072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845a718-d009-4fe9-954b-c2e424d21f7e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:02.150373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 796/1631] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:02.151433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845a718-d009-4fe9-954b-c2e424d21f7e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:02.167670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845a718-d009-4fe9-954b-c2e424d21f7e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:02.174085 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba45c6-c9fa-4859-88c4-db9bf4e63247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:02.177110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba45c6-c9fa-4859-88c4-db9bf4e63247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:02.177394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba45c6-c9fa-4859-88c4-db9bf4e63247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:02.177668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba45c6-c9fa-4859-88c4-db9bf4e63247 admin admin] PATH_INFO: `/v3/projects/134263c87f7e441c90e1059cf4027576/users/f8bf97b21005495cb1ca9aac9eef516a/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:02.180477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba45c6-c9fa-4859-88c4-db9bf4e63247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=134263c87f7e441c90e1059cf4027576, user_id=f8bf97b21005495cb1ca9aac9eef516a, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:02.185431 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba45c6-c9fa-4859-88c4-db9bf4e63247 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:02.201379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f050789-6537-4f93-a873-de732bb9394e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:02.202885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f050789-6537-4f93-a873-de732bb9394e service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:02.210642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 797/1632] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:02 2022] PUT /identity/v3/projects/134263c87f7e441c90e1059cf4027576/users/f8bf97b21005495cb1ca9aac9eef516a/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:02.212760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb2d4d-0bf9-4fb3-bcc7-30bdf0b6bde9 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:02.213023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb2d4d-0bf9-4fb3-bcc7-30bdf0b6bde9 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:02.213338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb2d4d-0bf9-4fb3-bcc7-30bdf0b6bde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:02.223576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 836/1633] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:02.260398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d3fb2d4d-0bf9-4fb3-bcc7-30bdf0b6bde9 None None] MFA Rules not processed for user `f8bf97b21005495cb1ca9aac9eef516a`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:02.262542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d3fb2d4d-0bf9-4fb3-bcc7-30bdf0b6bde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:02.585362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:02.585362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 798/1634] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:02 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:02.588229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f050789-6537-4f93-a873-de732bb9394e service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:02.588229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f050789-6537-4f93-a873-de732bb9394e service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:02.588229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e03963-8ab0-42ef-a850-bdea0f3b30cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:02.588229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e03963-8ab0-42ef-a850-bdea0f3b30cd service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:02.591929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e03963-8ab0-42ef-a850-bdea0f3b30cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:02.591929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e03963-8ab0-42ef-a850-bdea0f3b30cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:02.591929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e03963-8ab0-42ef-a850-bdea0f3b30cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:02.591929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e03963-8ab0-42ef-a850-bdea0f3b30cd service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:02.591929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 837/1635] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:02 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:02.598264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb2d4d-0bf9-4fb3-bcc7-30bdf0b6bde9 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:02.612574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb2d4d-0bf9-4fb3-bcc7-30bdf0b6bde9 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:02.622620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dae63-ff1b-46d5-8cf0-82af74b01b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:02.624464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dae63-ff1b-46d5-8cf0-82af74b01b7b service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:02.624757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dae63-ff1b-46d5-8cf0-82af74b01b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:02.625026 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dae63-ff1b-46d5-8cf0-82af74b01b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:02.747095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dae63-ff1b-46d5-8cf0-82af74b01b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:02.749281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dae63-ff1b-46d5-8cf0-82af74b01b7b service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:02.762893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 799/1636] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:02.779638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c31f5-27bd-4627-88e8-f6690a2f0dc2 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:02.779638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c31f5-27bd-4627-88e8-f6690a2f0dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:02.779638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c31f5-27bd-4627-88e8-f6690a2f0dc2 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:02.779638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 838/1637] 10.222.0.58 () {54 vars in 962 bytes} [Mon Aug 29 23:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:51:02.785530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f4895-85c2-4129-8b42-4ff6de6cee54 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:02.786656 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f4895-85c2-4129-8b42-4ff6de6cee54 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:02.787076 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f4895-85c2-4129-8b42-4ff6de6cee54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:02.823821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:02.825075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d22f4895-85c2-4129-8b42-4ff6de6cee54 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:02.826702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d22f4895-85c2-4129-8b42-4ff6de6cee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:02.851227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c31f5-27bd-4627-88e8-f6690a2f0dc2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:02.923154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 800/1638] 10.222.0.58 () {58 vars in 1059 bytes} [Mon Aug 29 23:51:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:02.928892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebe861-fc2b-454a-9f36-85502c73b963 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:02.929193 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebe861-fc2b-454a-9f36-85502c73b963 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:02.929463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebe861-fc2b-454a-9f36-85502c73b963 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:02.935557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 801/1639] 10.222.0.58 () {54 vars in 912 bytes} [Mon Aug 29 23:51:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:51:02.936944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fe93a-458c-4a85-ac6d-536a01841082 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:02.937230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fe93a-458c-4a85-ac6d-536a01841082 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:02.937515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fe93a-458c-4a85-ac6d-536a01841082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:02.964453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-bb1fe93a-458c-4a85-ac6d-536a01841082 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:02.966061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bb1fe93a-458c-4a85-ac6d-536a01841082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:02.982731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 802/1640] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:51:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:02.989394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c31f5-27bd-4627-88e8-f6690a2f0dc2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:02.994904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:02.995543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:02.995823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:02.996100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] PATH_INFO: `/v3/users/8402373cf55e47fbb965123895642d70` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:02.997492 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8402373cf55e47fbb965123895642d70)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:02.998351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fe93a-458c-4a85-ac6d-536a01841082 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:02.999501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:03.010588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fe93a-458c-4a85-ac6d-536a01841082 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:03.017664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2c83d-48da-457e-b3da-64d5099eee7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:03.020527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2c83d-48da-457e-b3da-64d5099eee7f service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:03.020976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2c83d-48da-457e-b3da-64d5099eee7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:03.026924 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2c83d-48da-457e-b3da-64d5099eee7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:03.041727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2c83d-48da-457e-b3da-64d5099eee7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:03.043479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2c83d-48da-457e-b3da-64d5099eee7f service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:03.062782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 803/1641] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:51:03.243181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f10c9-9984-488d-84b9-d0e8ea6f796a None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:03.243627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f10c9-9984-488d-84b9-d0e8ea6f796a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:03.244542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f10c9-9984-488d-84b9-d0e8ea6f796a None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:03.246099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 804/1642] 10.222.0.58 () {54 vars in 912 bytes} [Mon Aug 29 23:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:51:03.251306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d558470-ffee-45ed-8d42-3cd3924f6b54 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:03.251652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d558470-ffee-45ed-8d42-3cd3924f6b54 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:03.252033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d558470-ffee-45ed-8d42-3cd3924f6b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:03.292785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8402373cf55e47fbb965123895642d70'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:03.295302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-8d558470-ffee-45ed-8d42-3cd3924f6b54 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:03.296607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8d558470-ffee-45ed-8d42-3cd3924f6b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:03.323318 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8402373cf55e47fbb965123895642d70'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:03.331817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 805/1643] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:51:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:03.341092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d558470-ffee-45ed-8d42-3cd3924f6b54 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:03.351326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8402373cf55e47fbb965123895642d70'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:03.357152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d558470-ffee-45ed-8d42-3cd3924f6b54 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:03.364548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58065980-55ae-4cf5-bf0b-bda9733a9be8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:03.365207 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065980-55ae-4cf5-bf0b-bda9733a9be8 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:03.365801 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065980-55ae-4cf5-bf0b-bda9733a9be8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:03.366088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065980-55ae-4cf5-bf0b-bda9733a9be8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:03.388311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58065980-55ae-4cf5-bf0b-bda9733a9be8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:03.388311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58065980-55ae-4cf5-bf0b-bda9733a9be8 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:03.409932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 806/1644] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:51:03.440506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8402373cf55e47fbb965123895642d70'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:03.441245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8402373cf55e47fbb965123895642d70'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:03.449105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 839/1645] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:02 2022] DELETE /identity/v3/users/8402373cf55e47fbb965123895642d70 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:03.456178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ed1c9-9ed4-4f6a-8901-ffe2ef93e4fc None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:03.456476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ed1c9-9ed4-4f6a-8901-ffe2ef93e4fc None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:03.456754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ed1c9-9ed4-4f6a-8901-ffe2ef93e4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:03.489654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f66ed1c9-9ed4-4f6a-8901-ffe2ef93e4fc None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:03.514655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f66ed1c9-9ed4-4f6a-8901-ffe2ef93e4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:03.523765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:03.525539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:03.629985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:03.670404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f04d9-32ba-40ac-98b9-c1aaa60a0a17 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:03.671349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 807/1646] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:03.675451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:03.678159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ed1c9-9ed4-4f6a-8901-ffe2ef93e4fc None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:03.687208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f66ed1c9-9ed4-4f6a-8901-ffe2ef93e4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:03.687521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:03.687521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:03.687521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:03.687521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] PATH_INFO: `/v3/users/76ab113b2f3144da87bf572cc0b2344f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:03.687804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=76ab113b2f3144da87bf572cc0b2344f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:03.687804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:03.761332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ed1c9-9ed4-4f6a-8901-ffe2ef93e4fc None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:03.765644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0830f30-801b-4cc6-85cf-cbb35d55b4de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:03.766455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0830f30-801b-4cc6-85cf-cbb35d55b4de service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:03.766744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0830f30-801b-4cc6-85cf-cbb35d55b4de service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:03.767017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0830f30-801b-4cc6-85cf-cbb35d55b4de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:03.778155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0830f30-801b-4cc6-85cf-cbb35d55b4de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:03.779799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0830f30-801b-4cc6-85cf-cbb35d55b4de service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:03.797485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 808/1647] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:51:03.980021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76ab113b2f3144da87bf572cc0b2344f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:04.019518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76ab113b2f3144da87bf572cc0b2344f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:04.031370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76ab113b2f3144da87bf572cc0b2344f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:04.127168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76ab113b2f3144da87bf572cc0b2344f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:04.133623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76ab113b2f3144da87bf572cc0b2344f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:04.166157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 840/1648] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:03 2022] DELETE /identity/v3/users/76ab113b2f3144da87bf572cc0b2344f => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:04.170866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07e211-351f-453b-97cf-6659ef31e446 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:04.171146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07e211-351f-453b-97cf-6659ef31e446 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:04.171418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07e211-351f-453b-97cf-6659ef31e446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:04.203210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-3b07e211-351f-453b-97cf-6659ef31e446 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:04.204548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3b07e211-351f-453b-97cf-6659ef31e446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:04.330613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 809/1649] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:04.343534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:04.353005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:04.442688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da5ae3-be39-476f-a198-5a959fcaae2a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:04.442688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987cf2e-8362-4d02-8fcf-91aa8d8caea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:04.442688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987cf2e-8362-4d02-8fcf-91aa8d8caea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:04.443256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987cf2e-8362-4d02-8fcf-91aa8d8caea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:04.443488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987cf2e-8362-4d02-8fcf-91aa8d8caea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:04.454667 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987cf2e-8362-4d02-8fcf-91aa8d8caea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:04.457127 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987cf2e-8362-4d02-8fcf-91aa8d8caea1 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:04.467986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 841/1650] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:04.548806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07e211-351f-453b-97cf-6659ef31e446 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:04.548806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3b07e211-351f-453b-97cf-6659ef31e446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:04.564677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07e211-351f-453b-97cf-6659ef31e446 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:04.571150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:04.571818 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:04.572192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:04.572564 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] PATH_INFO: `/v3/projects/49634e29b4934624a8b94d5385098c12` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:04.574018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49634e29b4934624a8b94d5385098c12)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:04.575955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:04.741645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49634e29b4934624a8b94d5385098c12', 'request_id': 'req-67424e85-cc58-4150-ad77-c59fec6a2152'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:04.905521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49634e29b4934624a8b94d5385098c12', 'request_id': 'req-67424e85-cc58-4150-ad77-c59fec6a2152'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:04.917784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49634e29b4934624a8b94d5385098c12', 'request_id': 'req-67424e85-cc58-4150-ad77-c59fec6a2152'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:04.949098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] The token cache is being invalidate because project 49634e29b4934624a8b94d5385098c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:04.949432 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:04.950388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:04.955127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 810/1651] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:04 2022] DELETE /identity/v3/projects/49634e29b4934624a8b94d5385098c12 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:04.957817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987cf2e-8362-4d02-8fcf-91aa8d8caea1 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:05.012595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987cf2e-8362-4d02-8fcf-91aa8d8caea1 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:05.170542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:05.171424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:05.171806 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:05.172780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] PATH_INFO: `/v3/users/f8bf97b21005495cb1ca9aac9eef516a` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:05.174400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8bf97b21005495cb1ca9aac9eef516a)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:05.177422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:05.386520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:05.410889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8bf97b21005495cb1ca9aac9eef516a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:05.448285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8bf97b21005495cb1ca9aac9eef516a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:05.466686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8bf97b21005495cb1ca9aac9eef516a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:05.537406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8bf97b21005495cb1ca9aac9eef516a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:05.540167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8bf97b21005495cb1ca9aac9eef516a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:05.553645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 842/1652] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:04 2022] DELETE /identity/v3/users/f8bf97b21005495cb1ca9aac9eef516a => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:06.435809 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:06.438753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:06.452924 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:06.469299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67424e85-cc58-4150-ad77-c59fec6a2152 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:06.573730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:06.574574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:06.574953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:06.575336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] PATH_INFO: `/v3/projects/f71b0d227a5147218377cab6f22a9b52` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:06.577645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71b0d227a5147218377cab6f22a9b52)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:06.582325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:06.731578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:06.738842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:06.764518 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71b0d227a5147218377cab6f22a9b52', 'request_id': 'req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:06.786239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281a8d5-c794-417a-a687-74379eebf1e1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:06.891195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71b0d227a5147218377cab6f22a9b52', 'request_id': 'req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:06.906847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71b0d227a5147218377cab6f22a9b52', 'request_id': 'req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:06.927272 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:06.928548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:06.929100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:06.937392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] PATH_INFO: `/v3/projects/134263c87f7e441c90e1059cf4027576` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:06.939297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=134263c87f7e441c90e1059cf4027576)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:06.942079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] The token cache is being invalidate because project f71b0d227a5147218377cab6f22a9b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:06.942234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:51:06.942234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 29 23:51:06.942234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: Aug 29 23:51:06.946318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:06.946559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:06.947190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-f8fffc5a-5f92-4a34-b9b1-9f5ccc1c0a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:06.955432 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 811/1653] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:06 2022] DELETE /identity/v3/projects/f71b0d227a5147218377cab6f22a9b52 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:06.963882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d13f2d-02b9-49ba-816d-c57ff13bdf24 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:06.964448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d13f2d-02b9-49ba-816d-c57ff13bdf24 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:06.964792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d13f2d-02b9-49ba-816d-c57ff13bdf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:07.021605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d6d13f2d-02b9-49ba-816d-c57ff13bdf24 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:07.022871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d6d13f2d-02b9-49ba-816d-c57ff13bdf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:07.058083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 812/1654] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:07.060811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26420f55-ef10-4d29-ae08-47852b1bc941 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:07.061261 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26420f55-ef10-4d29-ae08-47852b1bc941 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:07.061974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26420f55-ef10-4d29-ae08-47852b1bc941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:07.139474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-26420f55-ef10-4d29-ae08-47852b1bc941 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:07.147611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-26420f55-ef10-4d29-ae08-47852b1bc941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:07.199708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 813/1655] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:07.204668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26420f55-ef10-4d29-ae08-47852b1bc941 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:07.251664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26420f55-ef10-4d29-ae08-47852b1bc941 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:07.270151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfa38d-25a1-4259-9bbd-631e308ffda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:07.271136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfa38d-25a1-4259-9bbd-631e308ffda4 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:07.271623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfa38d-25a1-4259-9bbd-631e308ffda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:07.272776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfa38d-25a1-4259-9bbd-631e308ffda4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:07.273878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfa38d-25a1-4259-9bbd-631e308ffda4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:51:07.274395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfa38d-25a1-4259-9bbd-631e308ffda4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:07.276337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfa38d-25a1-4259-9bbd-631e308ffda4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:07.299054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 814/1656] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:51:07.304359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfa38d-25a1-4259-9bbd-631e308ffda4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:07.316372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfa38d-25a1-4259-9bbd-631e308ffda4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:07.325632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:07.370646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '134263c87f7e441c90e1059cf4027576', 'request_id': 'req-190a916e-5121-4ba3-a6d3-d439f624bf8a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:07.402035 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '134263c87f7e441c90e1059cf4027576', 'request_id': 'req-190a916e-5121-4ba3-a6d3-d439f624bf8a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:07.612599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da829743-9c34-42d2-a23f-ec874b9788ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:07.613336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da829743-9c34-42d2-a23f-ec874b9788ba admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:07.613845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da829743-9c34-42d2-a23f-ec874b9788ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:07.614133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da829743-9c34-42d2-a23f-ec874b9788ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:07.614907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da829743-9c34-42d2-a23f-ec874b9788ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:07.616979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da829743-9c34-42d2-a23f-ec874b9788ba admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:07.975296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '134263c87f7e441c90e1059cf4027576', 'request_id': 'req-190a916e-5121-4ba3-a6d3-d439f624bf8a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:08.000780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 815/1657] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:07 2022] POST /identity/v3/projects => generated 375 bytes in 697 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:51:08.001767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da829743-9c34-42d2-a23f-ec874b9788ba admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:08.032409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] The token cache is being invalidate because project 134263c87f7e441c90e1059cf4027576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:08.040497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:08.040497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:08.040497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 843/1658] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:06 2022] DELETE /identity/v3/projects/134263c87f7e441c90e1059cf4027576 => generated 0 bytes in 1307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:08.040497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:08.040497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:08.064376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a916e-5121-4ba3-a6d3-d439f624bf8a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:08.143757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:08.145222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:08.145664 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:08.146113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] PATH_INFO: `/v3/users/a102531e70e243469ce2b84e71b2a9b9` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:08.146230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da829743-9c34-42d2-a23f-ec874b9788ba admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:08.147800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a102531e70e243469ce2b84e71b2a9b9)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:08.150302 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:08.179954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7352a6a-8c1c-4cb8-99ef-a6affa3e2edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:08.180778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7352a6a-8c1c-4cb8-99ef-a6affa3e2edf admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:08.181294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7352a6a-8c1c-4cb8-99ef-a6affa3e2edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:08.182723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7352a6a-8c1c-4cb8-99ef-a6affa3e2edf admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:08.184707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7352a6a-8c1c-4cb8-99ef-a6affa3e2edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:08.193839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7352a6a-8c1c-4cb8-99ef-a6affa3e2edf admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:08.338577 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 816/1659] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:08 2022] POST /identity/v3/users => generated 288 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:08.339886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c045f7-2105-4f08-afe4-b0250675e0d9 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:08.340178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c045f7-2105-4f08-afe4-b0250675e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:08.340456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c045f7-2105-4f08-afe4-b0250675e0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:08.387052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a102531e70e243469ce2b84e71b2a9b9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:08.431261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a102531e70e243469ce2b84e71b2a9b9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:08.445175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-71c045f7-2105-4f08-afe4-b0250675e0d9 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:08.449902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-71c045f7-2105-4f08-afe4-b0250675e0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:08.451710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a102531e70e243469ce2b84e71b2a9b9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:08.462895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 817/1660] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:08.474794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c045f7-2105-4f08-afe4-b0250675e0d9 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:08.474794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-71c045f7-2105-4f08-afe4-b0250675e0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:08.487198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c045f7-2105-4f08-afe4-b0250675e0d9 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:08.494131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492db88b-13bf-44c1-b7f9-58d5fdae66c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:08.494927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492db88b-13bf-44c1-b7f9-58d5fdae66c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:08.495216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492db88b-13bf-44c1-b7f9-58d5fdae66c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:08.495538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492db88b-13bf-44c1-b7f9-58d5fdae66c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:08.496195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492db88b-13bf-44c1-b7f9-58d5fdae66c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:08.497794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492db88b-13bf-44c1-b7f9-58d5fdae66c6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:08.512958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 818/1661] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:08.514331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259c494-03e3-43be-b5e6-766fc6c12966 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:08.514650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259c494-03e3-43be-b5e6-766fc6c12966 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:08.514921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259c494-03e3-43be-b5e6-766fc6c12966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:08.549737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7259c494-03e3-43be-b5e6-766fc6c12966 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:08.549737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7259c494-03e3-43be-b5e6-766fc6c12966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:08.561080 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 819/1662] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:08.561811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259c494-03e3-43be-b5e6-766fc6c12966 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:08.566723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a102531e70e243469ce2b84e71b2a9b9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:08.567358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a102531e70e243469ce2b84e71b2a9b9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:08.575143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 844/1663] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:08 2022] DELETE /identity/v3/users/a102531e70e243469ce2b84e71b2a9b9 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:08.576348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:08.579747 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:08.585977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259c494-03e3-43be-b5e6-766fc6c12966 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:08.704382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:08.716265 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb10d9-8152-427d-add2-3ee49f76655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:08.716884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb10d9-8152-427d-add2-3ee49f76655c admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:08.717157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb10d9-8152-427d-add2-3ee49f76655c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:08.719523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb10d9-8152-427d-add2-3ee49f76655c admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:08.720220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb10d9-8152-427d-add2-3ee49f76655c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:08.725634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb10d9-8152-427d-add2-3ee49f76655c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:08.726566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a1b43-eadc-4be3-ba59-f588d9ee2d8e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:08.733673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4e387-a254-48f5-a359-46f709791f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:08.734051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4e387-a254-48f5-a359-46f709791f4a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:08.734336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4e387-a254-48f5-a359-46f709791f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:08.737614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4e387-a254-48f5-a359-46f709791f4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:08.737614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4e387-a254-48f5-a359-46f709791f4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:51:08.737614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4e387-a254-48f5-a359-46f709791f4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:08.739035 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4e387-a254-48f5-a359-46f709791f4a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:08.751404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 820/1664] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:08.755426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e82c8e-ddd6-4643-8a37-83aac2ffec0b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:08.755715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e82c8e-ddd6-4643-8a37-83aac2ffec0b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:08.755997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e82c8e-ddd6-4643-8a37-83aac2ffec0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:08.761390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 845/1665] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:51:08.766301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4e387-a254-48f5-a359-46f709791f4a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:08.766301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-33e4e387-a254-48f5-a359-46f709791f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:08.781648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4e387-a254-48f5-a359-46f709791f4a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:08.794342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e827034-1c89-4ba2-9d02-314df177dda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:08.795262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827034-1c89-4ba2-9d02-314df177dda4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:08.795895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827034-1c89-4ba2-9d02-314df177dda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:08.798095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827034-1c89-4ba2-9d02-314df177dda4 admin admin] PATH_INFO: `/v3/projects/e7c9a7f5b50547d682e8db3e114a7b9c/users/4f4620effdc8492da36fedeb3c5812bd/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:08.802403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-06e82c8e-ddd6-4643-8a37-83aac2ffec0b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:08.807278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-06e82c8e-ddd6-4643-8a37-83aac2ffec0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:08.833337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 821/1666] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:08.835892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e82c8e-ddd6-4643-8a37-83aac2ffec0b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:08.842820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e827034-1c89-4ba2-9d02-314df177dda4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c9a7f5b50547d682e8db3e114a7b9c, user_id=4f4620effdc8492da36fedeb3c5812bd, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:08.847939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e827034-1c89-4ba2-9d02-314df177dda4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:08.859873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e82c8e-ddd6-4643-8a37-83aac2ffec0b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:08.864681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115b01-a9ef-4e3a-be40-e3b0a1715f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:08.865386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115b01-a9ef-4e3a-be40-e3b0a1715f22 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:08.865721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115b01-a9ef-4e3a-be40-e3b0a1715f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:08.866137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115b01-a9ef-4e3a-be40-e3b0a1715f22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:08.867058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70115b01-a9ef-4e3a-be40-e3b0a1715f22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:08.870956 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70115b01-a9ef-4e3a-be40-e3b0a1715f22 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:09.004058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 846/1667] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:08 2022] PUT /identity/v3/projects/e7c9a7f5b50547d682e8db3e114a7b9c/users/4f4620effdc8492da36fedeb3c5812bd/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:09.006273 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e827034-1c89-4ba2-9d02-314df177dda4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:09.018934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0e827034-1c89-4ba2-9d02-314df177dda4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:09.034544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 822/1668] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:08 2022] POST /identity/v3/projects => generated 401 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:09.036003 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315972f1-2dfb-4510-99f4-1caa6f23d3d8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:09.036297 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315972f1-2dfb-4510-99f4-1caa6f23d3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:09.036577 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315972f1-2dfb-4510-99f4-1caa6f23d3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:09.170193 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-315972f1-2dfb-4510-99f4-1caa6f23d3d8 None None] MFA Rules not processed for user `4f4620effdc8492da36fedeb3c5812bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:09.171384 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-315972f1-2dfb-4510-99f4-1caa6f23d3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:09.212064 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e827034-1c89-4ba2-9d02-314df177dda4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:09.396476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 823/1669] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:09.397713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315972f1-2dfb-4510-99f4-1caa6f23d3d8 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:09.404476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbbb62-12f7-4a16-bfeb-3d96625edba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:09.407388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbbb62-12f7-4a16-bfeb-3d96625edba8 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:09.407610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbbb62-12f7-4a16-bfeb-3d96625edba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:09.407989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbbb62-12f7-4a16-bfeb-3d96625edba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:09.442154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:09.573620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315972f1-2dfb-4510-99f4-1caa6f23d3d8 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:09.586745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4621c44-f2ed-405e-b7c6-5a4315feeadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:09.587447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4621c44-f2ed-405e-b7c6-5a4315feeadb admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:09.587728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4621c44-f2ed-405e-b7c6-5a4315feeadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:09.587990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4621c44-f2ed-405e-b7c6-5a4315feeadb admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:09.588779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4621c44-f2ed-405e-b7c6-5a4315feeadb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:09.593053 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4621c44-f2ed-405e-b7c6-5a4315feeadb admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:09.653084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbbb62-12f7-4a16-bfeb-3d96625edba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:09.655602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbbb62-12f7-4a16-bfeb-3d96625edba8 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:09.673629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 847/1670] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:09.675466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbbb62-12f7-4a16-bfeb-3d96625edba8 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:09.707000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbbb62-12f7-4a16-bfeb-3d96625edba8 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:09.729502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe203101-fca3-437c-b712-9056077dcc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:09.732782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe203101-fca3-437c-b712-9056077dcc39 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:09.734522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe203101-fca3-437c-b712-9056077dcc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:09.735672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe203101-fca3-437c-b712-9056077dcc39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:09.738748 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe203101-fca3-437c-b712-9056077dcc39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:09.740757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe203101-fca3-437c-b712-9056077dcc39 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:09.747852 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 824/1671] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:09 2022] POST /identity/v3/users => generated 301 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:09.754579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4621c44-f2ed-405e-b7c6-5a4315feeadb admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:09.781605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4621c44-f2ed-405e-b7c6-5a4315feeadb admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:09.802489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b383ea2-c6f4-425a-8ff1-c56f80597947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:09.803351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383ea2-c6f4-425a-8ff1-c56f80597947 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:09.803630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383ea2-c6f4-425a-8ff1-c56f80597947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:09.803902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383ea2-c6f4-425a-8ff1-c56f80597947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:09.811331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b383ea2-c6f4-425a-8ff1-c56f80597947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:09.813143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b383ea2-c6f4-425a-8ff1-c56f80597947 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:09.823760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:09.840808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 825/1672] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:09.844975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b383ea2-c6f4-425a-8ff1-c56f80597947 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:09.860567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 848/1673] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:09 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:09.865463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe203101-fca3-437c-b712-9056077dcc39 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:09.874358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b383ea2-c6f4-425a-8ff1-c56f80597947 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:10.018013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe203101-fca3-437c-b712-9056077dcc39 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:10.040568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396509e7-0445-4983-8451-fdd2c3947b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:10.040568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396509e7-0445-4983-8451-fdd2c3947b07 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:10.040568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396509e7-0445-4983-8451-fdd2c3947b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:10.040568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396509e7-0445-4983-8451-fdd2c3947b07 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:10.044994 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5c1a1-8651-4640-97ca-39009d54c2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:10.045638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5c1a1-8651-4640-97ca-39009d54c2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:10.045918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5c1a1-8651-4640-97ca-39009d54c2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:10.046024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396509e7-0445-4983-8451-fdd2c3947b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:10.046185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5c1a1-8651-4640-97ca-39009d54c2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:10.046873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5c1a1-8651-4640-97ca-39009d54c2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:10.048529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5c1a1-8651-4640-97ca-39009d54c2f3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:10.049645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396509e7-0445-4983-8451-fdd2c3947b07 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:10.076050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 826/1674] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:51:10.080319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5c1a1-8651-4640-97ca-39009d54c2f3 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:10.101801 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5c1a1-8651-4640-97ca-39009d54c2f3 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:10.111693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff90cd-3179-48bc-ba4f-ec42fba47190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:10.112648 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff90cd-3179-48bc-ba4f-ec42fba47190 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:10.112929 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff90cd-3179-48bc-ba4f-ec42fba47190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:10.113197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff90cd-3179-48bc-ba4f-ec42fba47190 admin admin] PATH_INFO: `/v3/projects/8ff4798ed37441e1a6861fa945ea532c/users/57aea9701e484e58a5e2d87c3044807d/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:10.154918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff90cd-3179-48bc-ba4f-ec42fba47190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff4798ed37441e1a6861fa945ea532c, user_id=57aea9701e484e58a5e2d87c3044807d, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:10.157911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff90cd-3179-48bc-ba4f-ec42fba47190 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:10.160495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 849/1675] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:09 2022] POST /identity/v3/users => generated 295 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:51:10.162557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396509e7-0445-4983-8451-fdd2c3947b07 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:10.181606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396509e7-0445-4983-8451-fdd2c3947b07 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:10.193680 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cff54-0e57-4f42-a954-f752e80c90aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:10.194399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cff54-0e57-4f42-a954-f752e80c90aa admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:10.194801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cff54-0e57-4f42-a954-f752e80c90aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:10.195178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cff54-0e57-4f42-a954-f752e80c90aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:10.195932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cff54-0e57-4f42-a954-f752e80c90aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:10.197657 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cff54-0e57-4f42-a954-f752e80c90aa admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:10.223808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 850/1676] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:10.228915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cff54-0e57-4f42-a954-f752e80c90aa admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:10.247060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 827/1677] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:10 2022] PUT /identity/v3/projects/8ff4798ed37441e1a6861fa945ea532c/users/57aea9701e484e58a5e2d87c3044807d/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:10.253754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cff54-0e57-4f42-a954-f752e80c90aa admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:10.263151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c52b71-3b40-4c4c-ae2b-2b36595bdeee None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:10.263493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c52b71-3b40-4c4c-ae2b-2b36595bdeee None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:10.263849 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c52b71-3b40-4c4c-ae2b-2b36595bdeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:10.277469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:10.321154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-a2c52b71-3b40-4c4c-ae2b-2b36595bdeee None None] MFA Rules not processed for user `57aea9701e484e58a5e2d87c3044807d`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:10.327036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a2c52b71-3b40-4c4c-ae2b-2b36595bdeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:10.382455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f41fb-359c-41c7-90b9-4f43ecfaa2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:10.383289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f41fb-359c-41c7-90b9-4f43ecfaa2af admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:10.384010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f41fb-359c-41c7-90b9-4f43ecfaa2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:10.384010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f41fb-359c-41c7-90b9-4f43ecfaa2af admin admin] PATH_INFO: `/v3/projects/e23dd36834a2481187621884d73a21b0/users/7e3f33b904f9482a91b181d4d967efad/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:10.434563 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f41fb-359c-41c7-90b9-4f43ecfaa2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23dd36834a2481187621884d73a21b0, user_id=7e3f33b904f9482a91b181d4d967efad, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:10.439109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f41fb-359c-41c7-90b9-4f43ecfaa2af admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:10.479236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 828/1678] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:10 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:10.481825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c52b71-3b40-4c4c-ae2b-2b36595bdeee None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:10.495704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c52b71-3b40-4c4c-ae2b-2b36595bdeee None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:10.505080 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:10.506540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d3230-8305-4e5f-a058-967a7abc730a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:10.507439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d3230-8305-4e5f-a058-967a7abc730a admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:10.507943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d3230-8305-4e5f-a058-967a7abc730a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:10.508711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d3230-8305-4e5f-a058-967a7abc730a admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:10.509847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d3230-8305-4e5f-a058-967a7abc730a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:10.512119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d3230-8305-4e5f-a058-967a7abc730a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:10.542756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 851/1679] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:10 2022] PUT /identity/v3/projects/e23dd36834a2481187621884d73a21b0/users/7e3f33b904f9482a91b181d4d967efad/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:10.543767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f41fb-359c-41c7-90b9-4f43ecfaa2af admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:10.619569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 829/1680] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:10 2022] POST /identity/v3/projects => generated 399 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:10.620573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d3230-8305-4e5f-a058-967a7abc730a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:10.644271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f41fb-359c-41c7-90b9-4f43ecfaa2af admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:10.771281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042ca40-65e6-43e6-b666-6f373726a9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:10.773220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ca40-65e6-43e6-b666-6f373726a9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:10.773730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ca40-65e6-43e6-b666-6f373726a9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:10.774216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ca40-65e6-43e6-b666-6f373726a9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:10.775141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042ca40-65e6-43e6-b666-6f373726a9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:10.776666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d3230-8305-4e5f-a058-967a7abc730a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:10.778238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042ca40-65e6-43e6-b666-6f373726a9d1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:10.784715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b29753-7537-4189-8b8b-d1ba8abbc6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:10.785534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29753-7537-4189-8b8b-d1ba8abbc6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:10.786016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29753-7537-4189-8b8b-d1ba8abbc6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:10.786295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29753-7537-4189-8b8b-d1ba8abbc6e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:10.787756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b29753-7537-4189-8b8b-d1ba8abbc6e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:10.790029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b29753-7537-4189-8b8b-d1ba8abbc6e1 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:10.799269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 852/1681] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:10.803511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042ca40-65e6-43e6-b666-6f373726a9d1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:10.822233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042ca40-65e6-43e6-b666-6f373726a9d1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:10.830406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a2506-358a-4293-b357-3e75cc4594dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:10.831652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2506-358a-4293-b357-3e75cc4594dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:10.832532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2506-358a-4293-b357-3e75cc4594dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:10.832920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2506-358a-4293-b357-3e75cc4594dc admin admin] PATH_INFO: `/v3/projects/e23dd36834a2481187621884d73a21b0/users/7e3f33b904f9482a91b181d4d967efad/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:10.835673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a2506-358a-4293-b357-3e75cc4594dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23dd36834a2481187621884d73a21b0, user_id=7e3f33b904f9482a91b181d4d967efad, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:10.839321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a2506-358a-4293-b357-3e75cc4594dc admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:10.864509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 853/1682] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:10 2022] PUT /identity/v3/projects/e23dd36834a2481187621884d73a21b0/users/7e3f33b904f9482a91b181d4d967efad/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:10.874904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c61dc-957e-4126-b57e-db955aa6b6f6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:10.886835 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c61dc-957e-4126-b57e-db955aa6b6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:10.886835 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c61dc-957e-4126-b57e-db955aa6b6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:10.915888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 830/1683] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:10 2022] POST /identity/v3/users => generated 306 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:10.919533 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b29753-7537-4189-8b8b-d1ba8abbc6e1 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:10.939446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b29753-7537-4189-8b8b-d1ba8abbc6e1 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:10.957215 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b52c61dc-957e-4126-b57e-db955aa6b6f6 None None] MFA Rules not processed for user `7e3f33b904f9482a91b181d4d967efad`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:10.962310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b52c61dc-957e-4126-b57e-db955aa6b6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:11.103151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914f35e-b658-4903-a111-128393bb923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:11.107035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914f35e-b658-4903-a111-128393bb923d admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:11.107380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914f35e-b658-4903-a111-128393bb923d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:11.107803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914f35e-b658-4903-a111-128393bb923d admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:11.108795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914f35e-b658-4903-a111-128393bb923d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:11.110651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914f35e-b658-4903-a111-128393bb923d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:11.147977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 831/1684] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:11.147977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914f35e-b658-4903-a111-128393bb923d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:11.154763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7914f35e-b658-4903-a111-128393bb923d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:11.191059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914f35e-b658-4903-a111-128393bb923d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:11.197173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 854/1685] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:51:11.201469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c61dc-957e-4126-b57e-db955aa6b6f6 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:11.201614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:11.207307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:11.210195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:11.210195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:11.210195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] PATH_INFO: `/v3/projects/fc38f27d21dc494d9039037c3286cf6e` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:11.219975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc38f27d21dc494d9039037c3286cf6e)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:11.223794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:11.225993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c61dc-957e-4126-b57e-db955aa6b6f6 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:11.231371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42af96-9fb7-4cb0-a543-e70067eb07d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:11.231995 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42af96-9fb7-4cb0-a543-e70067eb07d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:11.232271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42af96-9fb7-4cb0-a543-e70067eb07d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:11.232547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42af96-9fb7-4cb0-a543-e70067eb07d1 admin admin] PATH_INFO: `/v3/projects/25605943fb14406690b5924e55535b18/users/f8ce6ec438fa4000861150bf48145f3a/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:11.264553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42af96-9fb7-4cb0-a543-e70067eb07d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25605943fb14406690b5924e55535b18, user_id=f8ce6ec438fa4000861150bf48145f3a, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:11.264861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42af96-9fb7-4cb0-a543-e70067eb07d1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:11.378610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 855/1686] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:11 2022] PUT /identity/v3/projects/25605943fb14406690b5924e55535b18/users/f8ce6ec438fa4000861150bf48145f3a/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:11.388353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f469c7-87a4-4bb5-937c-39caa5c73916 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:11.388353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f469c7-87a4-4bb5-937c-39caa5c73916 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:11.388353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f469c7-87a4-4bb5-937c-39caa5c73916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:11.446121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc38f27d21dc494d9039037c3286cf6e', 'request_id': 'req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:11.487808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a1f469c7-87a4-4bb5-937c-39caa5c73916 None None] MFA Rules not processed for user `db604b7af623452fac9b3712b6978168`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:11.489208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a1f469c7-87a4-4bb5-937c-39caa5c73916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:11.492503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:11.532455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc38f27d21dc494d9039037c3286cf6e', 'request_id': 'req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:11.543827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc38f27d21dc494d9039037c3286cf6e', 'request_id': 'req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:11.569552 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] The token cache is being invalidate because project fc38f27d21dc494d9039037c3286cf6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:11.569936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:11.570660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:11.575143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 832/1687] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:11 2022] DELETE /identity/v3/projects/fc38f27d21dc494d9039037c3286cf6e => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:11.575143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:11.575143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:11.613103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 856/1688] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:11.614282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f469c7-87a4-4bb5-937c-39caa5c73916 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:11.616709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a1f469c7-87a4-4bb5-937c-39caa5c73916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:11.704273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f753b-2ac7-4e0e-96e5-a18de7d473e3 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:11.714163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096c4a3-91f3-4d4a-9924-3a5447bc55a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:11.714163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096c4a3-91f3-4d4a-9924-3a5447bc55a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:11.714163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096c4a3-91f3-4d4a-9924-3a5447bc55a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:11.714163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096c4a3-91f3-4d4a-9924-3a5447bc55a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:11.714548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3096c4a3-91f3-4d4a-9924-3a5447bc55a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:11.801870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096c4a3-91f3-4d4a-9924-3a5447bc55a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:11.803436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096c4a3-91f3-4d4a-9924-3a5447bc55a7 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:11.814832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f469c7-87a4-4bb5-937c-39caa5c73916 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:11.816982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 833/1689] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:11.829566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003fe19-9428-4ba6-9698-04a320baf0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:11.830121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096c4a3-91f3-4d4a-9924-3a5447bc55a7 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:11.830242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003fe19-9428-4ba6-9698-04a320baf0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:11.830486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003fe19-9428-4ba6-9698-04a320baf0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:11.834555 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003fe19-9428-4ba6-9698-04a320baf0b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:11.834555 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003fe19-9428-4ba6-9698-04a320baf0b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:11.834754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3096c4a3-91f3-4d4a-9924-3a5447bc55a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:11.836649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003fe19-9428-4ba6-9698-04a320baf0b5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:11.867445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 857/1690] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:11.868400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003fe19-9428-4ba6-9698-04a320baf0b5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:11.873621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096c4a3-91f3-4d4a-9924-3a5447bc55a7 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:11.873944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1003fe19-9428-4ba6-9698-04a320baf0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:11.887600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:11.888252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:11.888534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:11.888812 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] PATH_INFO: `/v3/users/05b38113df444c48898ace21dcea68db` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:11.890008 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b38113df444c48898ace21dcea68db)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:11.891054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: WARNING py.warnings [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 23:51:11.891054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: warnings.warn(msg) Aug 29 23:51:11.891054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: Aug 29 23:51:11.892292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:12.106372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003fe19-9428-4ba6-9698-04a320baf0b5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:12.127503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:12.133088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4eef71-f2a3-41e4-907b-5536c64ef6e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:12.136320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eef71-f2a3-41e4-907b-5536c64ef6e8 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:12.136320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eef71-f2a3-41e4-907b-5536c64ef6e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:12.136320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eef71-f2a3-41e4-907b-5536c64ef6e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:12.153057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4eef71-f2a3-41e4-907b-5536c64ef6e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:12.157894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4eef71-f2a3-41e4-907b-5536c64ef6e8 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:12.164042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b38113df444c48898ace21dcea68db'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:12.188768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 858/1691] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:12.197030 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4eef71-f2a3-41e4-907b-5536c64ef6e8 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:12.228316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4eef71-f2a3-41e4-907b-5536c64ef6e8 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:12.233475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b38113df444c48898ace21dcea68db'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:12.247862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34135062-23a6-436b-940b-635e094a14b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:12.254311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34135062-23a6-436b-940b-635e094a14b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:12.254311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34135062-23a6-436b-940b-635e094a14b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:12.254311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34135062-23a6-436b-940b-635e094a14b1 admin admin] PATH_INFO: `/v3/projects/25605943fb14406690b5924e55535b18/users/f8ce6ec438fa4000861150bf48145f3a/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:12.257023 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34135062-23a6-436b-940b-635e094a14b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25605943fb14406690b5924e55535b18, user_id=f8ce6ec438fa4000861150bf48145f3a, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:12.259548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b38113df444c48898ace21dcea68db'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:12.262022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34135062-23a6-436b-940b-635e094a14b1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:12.309938 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 859/1692] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:12 2022] PUT /identity/v3/projects/25605943fb14406690b5924e55535b18/users/f8ce6ec438fa4000861150bf48145f3a/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:12.311844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceca39-2603-48d6-b7f7-803c22a0d2d2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:12.312135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceca39-2603-48d6-b7f7-803c22a0d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:12.312412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceca39-2603-48d6-b7f7-803c22a0d2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:12.344266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b38113df444c48898ace21dcea68db'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:12.349575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b38113df444c48898ace21dcea68db'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:12.354940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:12.362824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 834/1693] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:11 2022] DELETE /identity/v3/users/05b38113df444c48898ace21dcea68db => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:12.399923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a2ceca39-2603-48d6-b7f7-803c22a0d2d2 None None] MFA Rules not processed for user `f8ce6ec438fa4000861150bf48145f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:12.401673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a2ceca39-2603-48d6-b7f7-803c22a0d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:12.556917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 860/1694] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:12 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:12.575571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:12.578135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:12.681014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e59bf7-d9c3-4e01-8e10-688526fc3bd4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:12.687654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dd53c-6bf5-4d21-8162-6d836d791a14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:12.690998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dd53c-6bf5-4d21-8162-6d836d791a14 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:12.690998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dd53c-6bf5-4d21-8162-6d836d791a14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:12.690998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dd53c-6bf5-4d21-8162-6d836d791a14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:12.704523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dd53c-6bf5-4d21-8162-6d836d791a14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:12.707436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dd53c-6bf5-4d21-8162-6d836d791a14 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:12.732516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 835/1695] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:12 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:51:13.086933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceca39-2603-48d6-b7f7-803c22a0d2d2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:13.104873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceca39-2603-48d6-b7f7-803c22a0d2d2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:13.133124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7cc04-6ade-487d-96be-9e4e91d4c3ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:13.140251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7cc04-6ade-487d-96be-9e4e91d4c3ed service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:13.142462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7cc04-6ade-487d-96be-9e4e91d4c3ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:13.144452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7cc04-6ade-487d-96be-9e4e91d4c3ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:13.150534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-62b7cc04-6ade-487d-96be-9e4e91d4c3ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:13.436429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7cc04-6ade-487d-96be-9e4e91d4c3ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:13.438203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7cc04-6ade-487d-96be-9e4e91d4c3ed service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:13.473369 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 861/1696] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:13 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:13.509337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:14.090921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dd53c-6bf5-4d21-8162-6d836d791a14 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:14.093057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-de4dd53c-6bf5-4d21-8162-6d836d791a14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:14.262554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dd53c-6bf5-4d21-8162-6d836d791a14 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:14.281174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:14.281174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:14.281174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:14.281174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] PATH_INFO: `/v3/projects/803ce2e918ff438fbb4aa74065c37cf8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:14.281730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] RBAC: Authorizing `identity:delete_project(project_id=803ce2e918ff438fbb4aa74065c37cf8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:14.281730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:14.480450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '803ce2e918ff438fbb4aa74065c37cf8', 'request_id': 'req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:14.490202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:14.572136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '803ce2e918ff438fbb4aa74065c37cf8', 'request_id': 'req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:14.579205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '803ce2e918ff438fbb4aa74065c37cf8', 'request_id': 'req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:14.602345 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] The token cache is being invalidate because project 803ce2e918ff438fbb4aa74065c37cf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:14.604209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:14.604671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d5b16c7a-3806-42d1-bdcb-9fce91c9360d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:14.615064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 836/1697] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:14 2022] DELETE /identity/v3/projects/803ce2e918ff438fbb4aa74065c37cf8 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:14.616780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbbf24-29d5-4d70-9475-592f53ce0880 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:14.617285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbbf24-29d5-4d70-9475-592f53ce0880 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:14.617587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbbf24-29d5-4d70-9475-592f53ce0880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:14.660950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f9bbbf24-29d5-4d70-9475-592f53ce0880 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:14.663726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f9bbbf24-29d5-4d70-9475-592f53ce0880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:14.774498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 862/1698] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:14.777754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13f43f-3771-46fd-9007-6d15efc3c422 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:14.778091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13f43f-3771-46fd-9007-6d15efc3c422 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:14.778365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13f43f-3771-46fd-9007-6d15efc3c422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:14.802516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-4d13f43f-3771-46fd-9007-6d15efc3c422 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:14.803862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4d13f43f-3771-46fd-9007-6d15efc3c422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:14.820438 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 837/1699] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:14.821406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbbf24-29d5-4d70-9475-592f53ce0880 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:14.835455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbbf24-29d5-4d70-9475-592f53ce0880 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:14.843409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e770d-43cd-4bf6-ab86-977e89c65355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:14.844987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e770d-43cd-4bf6-ab86-977e89c65355 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:14.845528 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e770d-43cd-4bf6-ab86-977e89c65355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:14.846006 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e770d-43cd-4bf6-ab86-977e89c65355 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:14.847114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e770d-43cd-4bf6-ab86-977e89c65355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:51:14.847961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e770d-43cd-4bf6-ab86-977e89c65355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:14.850004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e770d-43cd-4bf6-ab86-977e89c65355 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:14.870686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13f43f-3771-46fd-9007-6d15efc3c422 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:14.877497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 863/1700] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:51:14.885553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13f43f-3771-46fd-9007-6d15efc3c422 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:14.891392 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfb73b-2df9-492c-960a-a86708555d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:14.892361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfb73b-2df9-492c-960a-a86708555d11 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:14.892911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfb73b-2df9-492c-960a-a86708555d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:14.893980 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfb73b-2df9-492c-960a-a86708555d11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:14.895599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfb73b-2df9-492c-960a-a86708555d11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:14.898071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfb73b-2df9-492c-960a-a86708555d11 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:15.024818 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 838/1701] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:14 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:51:15.029181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e770d-43cd-4bf6-ab86-977e89c65355 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:15.062727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e770d-43cd-4bf6-ab86-977e89c65355 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:15.247643 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7be8ef-4e85-4113-8679-473065638ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:15.248263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be8ef-4e85-4113-8679-473065638ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:15.250994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be8ef-4e85-4113-8679-473065638ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:15.251418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be8ef-4e85-4113-8679-473065638ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:15.252564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7be8ef-4e85-4113-8679-473065638ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:15.261071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7be8ef-4e85-4113-8679-473065638ca5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:15.288238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:15.416688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 864/1702] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:15 2022] POST /identity/v3/users => generated 294 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:51:15.420689 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfb73b-2df9-492c-960a-a86708555d11 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:15.472179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfb73b-2df9-492c-960a-a86708555d11 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:15.511056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadadd89-2097-4ff8-a0bc-34aad3cb3293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:15.512528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadadd89-2097-4ff8-a0bc-34aad3cb3293 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:15.513054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadadd89-2097-4ff8-a0bc-34aad3cb3293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:15.513533 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadadd89-2097-4ff8-a0bc-34aad3cb3293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:15.514718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadadd89-2097-4ff8-a0bc-34aad3cb3293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:15.516747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadadd89-2097-4ff8-a0bc-34aad3cb3293 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:15.554783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 839/1703] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:15.559694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7be8ef-4e85-4113-8679-473065638ca5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:15.601469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7be8ef-4e85-4113-8679-473065638ca5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:15.626290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba8016-5ce1-43e7-89d9-760964430d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:15.628890 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba8016-5ce1-43e7-89d9-760964430d53 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:15.630472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba8016-5ce1-43e7-89d9-760964430d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:15.631001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba8016-5ce1-43e7-89d9-760964430d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:15.635850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba8016-5ce1-43e7-89d9-760964430d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:15.637731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba8016-5ce1-43e7-89d9-760964430d53 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:15.678316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadadd89-2097-4ff8-a0bc-34aad3cb3293 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:15.687621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 865/1704] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:15.695746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadadd89-2097-4ff8-a0bc-34aad3cb3293 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:15.712459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f791526-fb7a-4b3a-ac94-c462bd11aba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:15.713166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f791526-fb7a-4b3a-ac94-c462bd11aba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:15.713448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f791526-fb7a-4b3a-ac94-c462bd11aba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:15.713722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f791526-fb7a-4b3a-ac94-c462bd11aba3 admin admin] PATH_INFO: `/v3/projects/41f152df6b7c4abcbed283081ea896d4/users/53e219567d914079af5d1e59ad9a89cb/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:15.763376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f791526-fb7a-4b3a-ac94-c462bd11aba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f152df6b7c4abcbed283081ea896d4, user_id=53e219567d914079af5d1e59ad9a89cb, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:15.766750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f791526-fb7a-4b3a-ac94-c462bd11aba3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:15.904181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 840/1705] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:15 2022] PUT /identity/v3/projects/41f152df6b7c4abcbed283081ea896d4/users/53e219567d914079af5d1e59ad9a89cb/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:15.910522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0d372-1853-407c-971f-d9f552e00343 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:15.911178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0d372-1853-407c-971f-d9f552e00343 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:15.913113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0d372-1853-407c-971f-d9f552e00343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:15.986221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1dc0d372-1853-407c-971f-d9f552e00343 None None] MFA Rules not processed for user `53e219567d914079af5d1e59ad9a89cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:15.988042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1dc0d372-1853-407c-971f-d9f552e00343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:16.174581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 866/1706] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:16.181088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f791526-fb7a-4b3a-ac94-c462bd11aba3 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:16.419956 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f791526-fb7a-4b3a-ac94-c462bd11aba3 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:16.454371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141294a0-e705-4537-b5db-ce2b25928f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:16.457766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141294a0-e705-4537-b5db-ce2b25928f27 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:16.457766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141294a0-e705-4537-b5db-ce2b25928f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:16.457766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141294a0-e705-4537-b5db-ce2b25928f27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:16.459626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141294a0-e705-4537-b5db-ce2b25928f27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:16.464075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141294a0-e705-4537-b5db-ce2b25928f27 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:16.577093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 841/1707] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:16 2022] POST /identity/v3/projects => generated 385 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:16.581395 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0d372-1853-407c-971f-d9f552e00343 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:16.602611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0d372-1853-407c-971f-d9f552e00343 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:16.748413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e08aa-1a63-4438-8806-4ac08bed355b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:16.749241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e08aa-1a63-4438-8806-4ac08bed355b admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:16.749611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e08aa-1a63-4438-8806-4ac08bed355b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:16.749886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e08aa-1a63-4438-8806-4ac08bed355b admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:16.750609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e08aa-1a63-4438-8806-4ac08bed355b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:16.759317 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e08aa-1a63-4438-8806-4ac08bed355b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:16.874795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 867/1708] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:16 2022] POST /identity/v3/users => generated 299 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:16.880668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141294a0-e705-4537-b5db-ce2b25928f27 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:16.903798 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141294a0-e705-4537-b5db-ce2b25928f27 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:16.918342 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a5ab8-0340-4982-a0a3-6eae029a4aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:16.919351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5ab8-0340-4982-a0a3-6eae029a4aca admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:16.919646 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5ab8-0340-4982-a0a3-6eae029a4aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:16.919977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5ab8-0340-4982-a0a3-6eae029a4aca admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:16.920638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a5ab8-0340-4982-a0a3-6eae029a4aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:16.935795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a5ab8-0340-4982-a0a3-6eae029a4aca admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:16.979985 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 842/1709] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:16.984144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e08aa-1a63-4438-8806-4ac08bed355b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:17.008738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e08aa-1a63-4438-8806-4ac08bed355b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:17.033518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8861f-b354-43ea-9955-8ad15e5a2aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:17.034452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8861f-b354-43ea-9955-8ad15e5a2aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:17.034880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8861f-b354-43ea-9955-8ad15e5a2aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:17.035324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8861f-b354-43ea-9955-8ad15e5a2aa1 admin admin] PATH_INFO: `/v3/projects/a4becd0c8ad14121aea462f08381d831/users/1fdfc79bac414e0b9ae15669de63d830/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:17.103731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8861f-b354-43ea-9955-8ad15e5a2aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4becd0c8ad14121aea462f08381d831, user_id=1fdfc79bac414e0b9ae15669de63d830, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:17.103731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8861f-b354-43ea-9955-8ad15e5a2aa1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:17.200702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 868/1710] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:16 2022] PUT /identity/v3/projects/a4becd0c8ad14121aea462f08381d831/users/1fdfc79bac414e0b9ae15669de63d830/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:17.205689 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a5ab8-0340-4982-a0a3-6eae029a4aca admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:17.239443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a5ab8-0340-4982-a0a3-6eae029a4aca admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:17.439951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:17.474537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27dda4-b53e-4a42-942c-4ae8d93025ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:17.475431 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27dda4-b53e-4a42-942c-4ae8d93025ac admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:17.475727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27dda4-b53e-4a42-942c-4ae8d93025ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:17.476009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27dda4-b53e-4a42-942c-4ae8d93025ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:17.484201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27dda4-b53e-4a42-942c-4ae8d93025ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:17.491463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27dda4-b53e-4a42-942c-4ae8d93025ac admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:17.524283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 843/1711] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:17.525863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8861f-b354-43ea-9955-8ad15e5a2aa1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:17.550284 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8861f-b354-43ea-9955-8ad15e5a2aa1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:17.576599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e6b04-a2de-4425-ac5c-1f82d7ff744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:17.577324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e6b04-a2de-4425-ac5c-1f82d7ff744a admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:17.577609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e6b04-a2de-4425-ac5c-1f82d7ff744a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:17.577886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e6b04-a2de-4425-ac5c-1f82d7ff744a admin admin] PATH_INFO: `/v3/projects/a4becd0c8ad14121aea462f08381d831/users/1fdfc79bac414e0b9ae15669de63d830/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:17.583619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e6b04-a2de-4425-ac5c-1f82d7ff744a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4becd0c8ad14121aea462f08381d831, user_id=1fdfc79bac414e0b9ae15669de63d830, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:17.586321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e6b04-a2de-4425-ac5c-1f82d7ff744a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:17.630514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 869/1712] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:17 2022] PUT /identity/v3/projects/a4becd0c8ad14121aea462f08381d831/users/1fdfc79bac414e0b9ae15669de63d830/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:17.637584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11b4e6-9a61-4edd-a8e8-a6195f78691c None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:17.637757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11b4e6-9a61-4edd-a8e8-a6195f78691c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:17.641246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11b4e6-9a61-4edd-a8e8-a6195f78691c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:17.716418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-1e11b4e6-9a61-4edd-a8e8-a6195f78691c None None] MFA Rules not processed for user `1fdfc79bac414e0b9ae15669de63d830`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:17.720384 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1e11b4e6-9a61-4edd-a8e8-a6195f78691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:17.961715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 844/1713] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:17.984361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e6b04-a2de-4425-ac5c-1f82d7ff744a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:17.986805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2a9e6b04-a2de-4425-ac5c-1f82d7ff744a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:18.078647 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:18.149141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e6b04-a2de-4425-ac5c-1f82d7ff744a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:18.158509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8f45f-d640-403d-8556-34fff4be5176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:18.158509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f45f-d640-403d-8556-34fff4be5176 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:18.158509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f45f-d640-403d-8556-34fff4be5176 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:18.158509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f45f-d640-403d-8556-34fff4be5176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:18.171430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8f45f-d640-403d-8556-34fff4be5176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:18.173867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8f45f-d640-403d-8556-34fff4be5176 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:18.203197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 870/1714] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:51:18.245711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11b4e6-9a61-4edd-a8e8-a6195f78691c None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:18.413350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11b4e6-9a61-4edd-a8e8-a6195f78691c None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:18.425206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:18.426857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:18.426857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:18.426857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] PATH_INFO: `/v3/users/53e219567d914079af5d1e59ad9a89cb` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:18.429600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e219567d914079af5d1e59ad9a89cb)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:18.436837 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:18.729527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e219567d914079af5d1e59ad9a89cb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:18.764813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e219567d914079af5d1e59ad9a89cb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:18.798404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e219567d914079af5d1e59ad9a89cb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:18.904242 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e219567d914079af5d1e59ad9a89cb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:18.904968 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e219567d914079af5d1e59ad9a89cb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:18.918349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 845/1715] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:18 2022] DELETE /identity/v3/users/53e219567d914079af5d1e59ad9a89cb => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:18.920133 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45781cde-661a-4395-9c8b-3d72beebd29f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:18.920424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45781cde-661a-4395-9c8b-3d72beebd29f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:18.920677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45781cde-661a-4395-9c8b-3d72beebd29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:18.983118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-45781cde-661a-4395-9c8b-3d72beebd29f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:18.984360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-45781cde-661a-4395-9c8b-3d72beebd29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:19.071084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 871/1716] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:19.140331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:19.140535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:19.240116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:19.298870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20214c96-06ee-4011-bc8f-8549d2fdc111 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:19.317091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7a570-cc3d-4f0e-b568-47ca3f5230d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:19.323480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7a570-cc3d-4f0e-b568-47ca3f5230d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:19.323480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7a570-cc3d-4f0e-b568-47ca3f5230d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:19.323480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7a570-cc3d-4f0e-b568-47ca3f5230d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:19.369316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7a570-cc3d-4f0e-b568-47ca3f5230d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:19.369316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7a570-cc3d-4f0e-b568-47ca3f5230d3 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:19.408353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 846/1717] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:20.021226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3171444-c6e3-412a-8aa6-a9f5b6ec63c0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:20.021519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3171444-c6e3-412a-8aa6-a9f5b6ec63c0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:20.021788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3171444-c6e3-412a-8aa6-a9f5b6ec63c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:20.071689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d3171444-c6e3-412a-8aa6-a9f5b6ec63c0 None None] MFA Rules not processed for user `734d44438bfb4a79a68aa6705e49b1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:20.074656 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d3171444-c6e3-412a-8aa6-a9f5b6ec63c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:20.239584 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 872/1718] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:20.250225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4440c1-6ab2-4680-adc6-60b02a0a0677 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:20.250795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4440c1-6ab2-4680-adc6-60b02a0a0677 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:20.250795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4440c1-6ab2-4680-adc6-60b02a0a0677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:20.314736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7e4440c1-6ab2-4680-adc6-60b02a0a0677 None None] MFA Rules not processed for user `734d44438bfb4a79a68aa6705e49b1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:20.316659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7e4440c1-6ab2-4680-adc6-60b02a0a0677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:20.329281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 847/1719] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:20.342878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3171444-c6e3-412a-8aa6-a9f5b6ec63c0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:20.344963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d3171444-c6e3-412a-8aa6-a9f5b6ec63c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:20.542009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:20.613713 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3171444-c6e3-412a-8aa6-a9f5b6ec63c0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:20.637814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfa501-4925-4272-9e6a-4aa9e627fe78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:20.640009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfa501-4925-4272-9e6a-4aa9e627fe78 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:20.640155 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfa501-4925-4272-9e6a-4aa9e627fe78 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:20.640691 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfa501-4925-4272-9e6a-4aa9e627fe78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:20.668222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfa501-4925-4272-9e6a-4aa9e627fe78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:20.672218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfa501-4925-4272-9e6a-4aa9e627fe78 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:20.689182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 873/1720] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:20.987940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4440c1-6ab2-4680-adc6-60b02a0a0677 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:20.990026 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7e4440c1-6ab2-4680-adc6-60b02a0a0677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:21.153176 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4440c1-6ab2-4680-adc6-60b02a0a0677 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:21.167380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d9890-da0e-43b7-8c0e-9c27f72ff35c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:21.168317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d9890-da0e-43b7-8c0e-9c27f72ff35c service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:21.168807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d9890-da0e-43b7-8c0e-9c27f72ff35c service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:21.169273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d9890-da0e-43b7-8c0e-9c27f72ff35c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:21.171623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b10d9890-da0e-43b7-8c0e-9c27f72ff35c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:21.175251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd32e9b-7f3c-4b65-8cca-ba2995595a1d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:21.179975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd32e9b-7f3c-4b65-8cca-ba2995595a1d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:21.179975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd32e9b-7f3c-4b65-8cca-ba2995595a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:21.219825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3dd32e9b-7f3c-4b65-8cca-ba2995595a1d None None] MFA Rules not processed for user `db604b7af623452fac9b3712b6978168`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:21.224784 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3dd32e9b-7f3c-4b65-8cca-ba2995595a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:21.360211 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d9890-da0e-43b7-8c0e-9c27f72ff35c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:21.363164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d9890-da0e-43b7-8c0e-9c27f72ff35c service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:21.387442 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 874/1721] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:51:21.391887 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd32e9b-7f3c-4b65-8cca-ba2995595a1d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:21.398902 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3dd32e9b-7f3c-4b65-8cca-ba2995595a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:21.406666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 848/1722] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:21.408272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411d257-437f-4d9e-8461-2f47ef09cb6b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:21.410512 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411d257-437f-4d9e-8461-2f47ef09cb6b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:21.412681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411d257-437f-4d9e-8461-2f47ef09cb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:21.462111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd32e9b-7f3c-4b65-8cca-ba2995595a1d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:21.483821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:21.483821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:21.484121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:21.484309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] PATH_INFO: `/v3/users/57aea9701e484e58a5e2d87c3044807d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:21.495557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57aea9701e484e58a5e2d87c3044807d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:21.497597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:21.520491 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-6411d257-437f-4d9e-8461-2f47ef09cb6b None None] MFA Rules not processed for user `2b88462a0d8d4e3fb4672a3f1d219c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:21.522277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6411d257-437f-4d9e-8461-2f47ef09cb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:21.657181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:21.870626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 849/1723] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:21.880425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5b449-263c-477f-a0b0-25b0d15a0847 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:21.880425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5b449-263c-477f-a0b0-25b0d15a0847 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:21.880425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5b449-263c-477f-a0b0-25b0d15a0847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:21.883359 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57aea9701e484e58a5e2d87c3044807d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:21.944341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57aea9701e484e58a5e2d87c3044807d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:21.958150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-25d5b449-263c-477f-a0b0-25b0d15a0847 None None] MFA Rules not processed for user `34ef7ee27c3948ac9df1ff94ce24618e`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:21.959842 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-25d5b449-263c-477f-a0b0-25b0d15a0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:21.975090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57aea9701e484e58a5e2d87c3044807d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:22.064182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57aea9701e484e58a5e2d87c3044807d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:22.065230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57aea9701e484e58a5e2d87c3044807d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:22.069785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 850/1724] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:22.080300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d97eb-3ec5-4fb9-9ace-330d769f5c67 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:22.081075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d97eb-3ec5-4fb9-9ace-330d769f5c67 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:22.081425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d97eb-3ec5-4fb9-9ace-330d769f5c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:22.096086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 875/1725] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:21 2022] DELETE /identity/v3/users/57aea9701e484e58a5e2d87c3044807d => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:22.098449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:22.106099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:22.113659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-e36d97eb-3ec5-4fb9-9ace-330d769f5c67 None None] MFA Rules not processed for user `2b88462a0d8d4e3fb4672a3f1d219c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:22.115175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e36d97eb-3ec5-4fb9-9ace-330d769f5c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:22.210306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 851/1726] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:22.212186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33067f0-9e7e-4f11-8c6a-1b2a2643e055 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:22.212917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33067f0-9e7e-4f11-8c6a-1b2a2643e055 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:22.213206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33067f0-9e7e-4f11-8c6a-1b2a2643e055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:22.247916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d33067f0-9e7e-4f11-8c6a-1b2a2643e055 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:22.249734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d33067f0-9e7e-4f11-8c6a-1b2a2643e055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:22.353171 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84182d4-06e8-49ea-8db4-b2e78b264540 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:22.363739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:22.363917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 852/1727] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:22.364328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:22.364328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:22.364328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] PATH_INFO: `/v3/projects/41f152df6b7c4abcbed283081ea896d4` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:22.365938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33067f0-9e7e-4f11-8c6a-1b2a2643e055 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:22.367776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d33067f0-9e7e-4f11-8c6a-1b2a2643e055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:22.370375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f152df6b7c4abcbed283081ea896d4)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:22.372596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:22.376088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:22.491091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33067f0-9e7e-4f11-8c6a-1b2a2643e055 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:22.496667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5e251-4807-4e09-a33e-92a7683c8726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:22.497448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5e251-4807-4e09-a33e-92a7683c8726 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:22.497848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5e251-4807-4e09-a33e-92a7683c8726 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:22.498463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5e251-4807-4e09-a33e-92a7683c8726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:22.512135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5e251-4807-4e09-a33e-92a7683c8726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:22.513998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5e251-4807-4e09-a33e-92a7683c8726 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:22.547141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 853/1728] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:22.562127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc401371-26f0-491d-8ddf-2925f2306f94 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:22.562729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc401371-26f0-491d-8ddf-2925f2306f94 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:22.563429 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc401371-26f0-491d-8ddf-2925f2306f94 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:22.565170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f152df6b7c4abcbed283081ea896d4', 'request_id': 'req-9cc9b64b-6d88-43fb-a58d-17b98095329e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:22.568459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:22.577288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f152df6b7c4abcbed283081ea896d4', 'request_id': 'req-9cc9b64b-6d88-43fb-a58d-17b98095329e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:22.591751 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 854/1729] 10.222.0.58 () {54 vars in 912 bytes} [Mon Aug 29 23:51:22 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:51:22.599024 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a379007-cd36-4d4e-b7d0-c7189e0d78be None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:22.605162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a379007-cd36-4d4e-b7d0-c7189e0d78be None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:22.605162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a379007-cd36-4d4e-b7d0-c7189e0d78be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:22.656885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f152df6b7c4abcbed283081ea896d4', 'request_id': 'req-9cc9b64b-6d88-43fb-a58d-17b98095329e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:22.721069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] The token cache is being invalidate because project 41f152df6b7c4abcbed283081ea896d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:22.730651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:22.734355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:22.741854 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 876/1730] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:22 2022] DELETE /identity/v3/projects/41f152df6b7c4abcbed283081ea896d4 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:22.746957 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:22.759666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-0a379007-cd36-4d4e-b7d0-c7189e0d78be None None] MFA Rules not processed for user `863bcfa6b8694182b9d163133a141907`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:22.761585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:22.763398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-0a379007-cd36-4d4e-b7d0-c7189e0d78be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:22.899708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 855/1731] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:51:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:51:22.929596 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a379007-cd36-4d4e-b7d0-c7189e0d78be None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:22.936099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-0a379007-cd36-4d4e-b7d0-c7189e0d78be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:22.954259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9b64b-6d88-43fb-a58d-17b98095329e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:22.960719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13231c8-ba41-4385-904d-819f94f850ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:22.961392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13231c8-ba41-4385-904d-819f94f850ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:22.961673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13231c8-ba41-4385-904d-819f94f850ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:22.961953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13231c8-ba41-4385-904d-819f94f850ac admin admin] PATH_INFO: `/v3/users/1fdfc79bac414e0b9ae15669de63d830` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:22.963216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13231c8-ba41-4385-904d-819f94f850ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fdfc79bac414e0b9ae15669de63d830)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:22.963705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a379007-cd36-4d4e-b7d0-c7189e0d78be None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:22.966163 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13231c8-ba41-4385-904d-819f94f850ac admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:22.968676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bad0a-d534-485e-98b9-58de3d91c627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:22.969456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bad0a-d534-485e-98b9-58de3d91c627 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:22.969746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bad0a-d534-485e-98b9-58de3d91c627 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:22.970361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bad0a-d534-485e-98b9-58de3d91c627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:22.994413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bad0a-d534-485e-98b9-58de3d91c627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:22.996917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bad0a-d534-485e-98b9-58de3d91c627 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:23.020781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 856/1732] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:23.328165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bad0a-d534-485e-98b9-58de3d91c627 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:23.332109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-481bad0a-d534-485e-98b9-58de3d91c627 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:23.339266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a13231c8-ba41-4385-904d-819f94f850ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fdfc79bac414e0b9ae15669de63d830'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:23.355898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a13231c8-ba41-4385-904d-819f94f850ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fdfc79bac414e0b9ae15669de63d830'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:23.371721 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a13231c8-ba41-4385-904d-819f94f850ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fdfc79bac414e0b9ae15669de63d830'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:23.379647 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bad0a-d534-485e-98b9-58de3d91c627 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:23.390285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:23.391761 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:23.392705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:23.393425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] PATH_INFO: `/v3/projects/8ff4798ed37441e1a6861fa945ea532c` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:23.396186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff4798ed37441e1a6861fa945ea532c)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:23.426796 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:23.498002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a13231c8-ba41-4385-904d-819f94f850ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fdfc79bac414e0b9ae15669de63d830'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:23.506794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a13231c8-ba41-4385-904d-819f94f850ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fdfc79bac414e0b9ae15669de63d830'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:23.511456 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 877/1733] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:22 2022] DELETE /identity/v3/users/1fdfc79bac414e0b9ae15669de63d830 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:23.528620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae138841-35bf-4284-8c8d-5fc7e4827376 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:23.528899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae138841-35bf-4284-8c8d-5fc7e4827376 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:23.529170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae138841-35bf-4284-8c8d-5fc7e4827376 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:23.531153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 878/1734] 10.222.0.58 () {54 vars in 912 bytes} [Mon Aug 29 23:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:51:23.544622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b5042-1e46-424f-9a12-b8d784011086 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:23.544622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b5042-1e46-424f-9a12-b8d784011086 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:23.544622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b5042-1e46-424f-9a12-b8d784011086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:23.586117 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f12b5042-1e46-424f-9a12-b8d784011086 None None] MFA Rules not processed for user `863bcfa6b8694182b9d163133a141907`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:23.589650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f12b5042-1e46-424f-9a12-b8d784011086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:23.641530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff4798ed37441e1a6861fa945ea532c', 'request_id': 'req-eb074eec-c686-4abc-aa0d-e0e04637dfcb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:23.734648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 879/1735] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:51:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:23.738777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff4798ed37441e1a6861fa945ea532c', 'request_id': 'req-eb074eec-c686-4abc-aa0d-e0e04637dfcb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:23.752317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff4798ed37441e1a6861fa945ea532c', 'request_id': 'req-eb074eec-c686-4abc-aa0d-e0e04637dfcb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:23.772064 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b5042-1e46-424f-9a12-b8d784011086 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:23.773682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f12b5042-1e46-424f-9a12-b8d784011086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:23.790755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b5042-1e46-424f-9a12-b8d784011086 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:23.807181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] The token cache is being invalidate because project 8ff4798ed37441e1a6861fa945ea532c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:23.807181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:23.808985 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:23.833157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 857/1736] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:23 2022] DELETE /identity/v3/projects/8ff4798ed37441e1a6861fa945ea532c => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:23.836475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:23.845535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:23.892213 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:23.905200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:24.015227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c730c-efdd-402c-93b3-d79c5513e78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:24.015872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c730c-efdd-402c-93b3-d79c5513e78b service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:24.016160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c730c-efdd-402c-93b3-d79c5513e78b service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:24.016514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c730c-efdd-402c-93b3-d79c5513e78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:24.034005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5c0c730c-efdd-402c-93b3-d79c5513e78b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:24.078257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c730c-efdd-402c-93b3-d79c5513e78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:24.082024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c730c-efdd-402c-93b3-d79c5513e78b service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:24.105505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 880/1737] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:24.121083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb074eec-c686-4abc-aa0d-e0e04637dfcb admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:24.141171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:24.145263 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:24.145770 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:24.146165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] PATH_INFO: `/v3/users/f8ce6ec438fa4000861150bf48145f3a` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:24.148500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ce6ec438fa4000861150bf48145f3a)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:24.156748 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:24.342124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c730c-efdd-402c-93b3-d79c5513e78b service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:24.343628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5c0c730c-efdd-402c-93b3-d79c5513e78b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:24.382969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ce6ec438fa4000861150bf48145f3a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:24.412581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ce6ec438fa4000861150bf48145f3a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:24.420636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ce6ec438fa4000861150bf48145f3a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:24.492231 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ce6ec438fa4000861150bf48145f3a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:24.492937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ce6ec438fa4000861150bf48145f3a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:24.510464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 858/1738] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:23 2022] DELETE /identity/v3/users/f8ce6ec438fa4000861150bf48145f3a => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:24.511442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:24.515613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:24.537112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c730c-efdd-402c-93b3-d79c5513e78b service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:24.718509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c981010-ec49-407a-9e56-43ca6d4b59b6 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:24.724540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:24.725278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:24.725563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:24.725835 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] PATH_INFO: `/v3/projects/a4becd0c8ad14121aea462f08381d831` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:24.727568 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4becd0c8ad14121aea462f08381d831)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:24.734954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:24.781771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed51cd-df3e-44c0-8938-94733d55507d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:24.781771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed51cd-df3e-44c0-8938-94733d55507d service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:24.781771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed51cd-df3e-44c0-8938-94733d55507d service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:24.781771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed51cd-df3e-44c0-8938-94733d55507d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:24.791210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b5ed51cd-df3e-44c0-8938-94733d55507d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:24.892085 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4becd0c8ad14121aea462f08381d831', 'request_id': 'req-a74fed28-aac2-458b-aa87-02862b33c9aa'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:24.914026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed51cd-df3e-44c0-8938-94733d55507d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:24.918414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed51cd-df3e-44c0-8938-94733d55507d service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:24.933692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 881/1739] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:24.934881 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed51cd-df3e-44c0-8938-94733d55507d service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:24.938091 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b5ed51cd-df3e-44c0-8938-94733d55507d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:24.967048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4becd0c8ad14121aea462f08381d831', 'request_id': 'req-a74fed28-aac2-458b-aa87-02862b33c9aa'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:24.982027 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4becd0c8ad14121aea462f08381d831', 'request_id': 'req-a74fed28-aac2-458b-aa87-02862b33c9aa'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:25.027070 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] The token cache is being invalidate because project a4becd0c8ad14121aea462f08381d831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:25.030679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:25.036246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a74fed28-aac2-458b-aa87-02862b33c9aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:25.040397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 859/1740] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:24 2022] DELETE /identity/v3/projects/a4becd0c8ad14121aea462f08381d831 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:25.041898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:25.048081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6833046-a067-4e70-881d-311b8c002917 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:25.059692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6833046-a067-4e70-881d-311b8c002917 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:25.060484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6833046-a067-4e70-881d-311b8c002917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:25.109516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:25.153899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-b6833046-a067-4e70-881d-311b8c002917 None None] MFA Rules not processed for user `90b2c6a74c29443b94d1577b39c7ce0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:25.157125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b6833046-a067-4e70-881d-311b8c002917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:25.161514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed51cd-df3e-44c0-8938-94733d55507d service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:25.397373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 860/1741] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:25.399845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e6bd3-ba1f-49b0-bca9-fd6340c2215e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:25.400513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e6bd3-ba1f-49b0-bca9-fd6340c2215e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:25.401135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e6bd3-ba1f-49b0-bca9-fd6340c2215e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:25.408232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5a29c-d16c-4562-9e8c-ca07ad34b925 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:25.408841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5a29c-d16c-4562-9e8c-ca07ad34b925 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:25.409118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5a29c-d16c-4562-9e8c-ca07ad34b925 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:25.410277 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5a29c-d16c-4562-9e8c-ca07ad34b925 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:25.418872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4bf5a29c-d16c-4562-9e8c-ca07ad34b925 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:25.467568 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-185e6bd3-ba1f-49b0-bca9-fd6340c2215e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:25.470242 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-185e6bd3-ba1f-49b0-bca9-fd6340c2215e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:25.470795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5a29c-d16c-4562-9e8c-ca07ad34b925 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:25.480835 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5a29c-d16c-4562-9e8c-ca07ad34b925 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:25.528802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 882/1742] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:25.530009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5a29c-d16c-4562-9e8c-ca07ad34b925 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:25.532677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4bf5a29c-d16c-4562-9e8c-ca07ad34b925 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:25.566804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 861/1743] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:25.568955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e6bd3-ba1f-49b0-bca9-fd6340c2215e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:25.571238 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-185e6bd3-ba1f-49b0-bca9-fd6340c2215e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:25.595679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:25.601521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e6bd3-ba1f-49b0-bca9-fd6340c2215e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:25.617543 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:25.618373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:25.618703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:25.618982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] PATH_INFO: `/v3/projects/25605943fb14406690b5924e55535b18` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:25.620399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25605943fb14406690b5924e55535b18)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:25.625924 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:25.711455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5a29c-d16c-4562-9e8c-ca07ad34b925 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:25.721616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f05b02-d278-4f3b-bf1d-4e662c649cab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:25.723168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f05b02-d278-4f3b-bf1d-4e662c649cab service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:25.723814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f05b02-d278-4f3b-bf1d-4e662c649cab service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:25.724188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f05b02-d278-4f3b-bf1d-4e662c649cab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:25.727831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e5f05b02-d278-4f3b-bf1d-4e662c649cab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:25.850825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25605943fb14406690b5924e55535b18', 'request_id': 'req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:25.917536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f05b02-d278-4f3b-bf1d-4e662c649cab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:25.929020 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f05b02-d278-4f3b-bf1d-4e662c649cab service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:25.940192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 883/1744] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:25.951162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25605943fb14406690b5924e55535b18', 'request_id': 'req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:25.959059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911e39f-127e-4dfd-9b9f-005c6a3de1fe None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:25.959059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911e39f-127e-4dfd-9b9f-005c6a3de1fe None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:25.959059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911e39f-127e-4dfd-9b9f-005c6a3de1fe None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:25.975199 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 884/1745] 10.222.0.58 () {54 vars in 961 bytes} [Mon Aug 29 23:51:25 2022] GET /identity => generated 270 bytes in 34 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:51:25.987936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea935-20f8-4a19-9dc9-f3f0fd1b8e05 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:25.987936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea935-20f8-4a19-9dc9-f3f0fd1b8e05 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:25.987936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea935-20f8-4a19-9dc9-f3f0fd1b8e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:25.988405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25605943fb14406690b5924e55535b18', 'request_id': 'req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:26.017367 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] The token cache is being invalidate because project 25605943fb14406690b5924e55535b18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:26.017909 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:26.021246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdceeba2-7f9c-4c0a-b4f9-13b2b73c2872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:26.032212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 862/1746] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:25 2022] DELETE /identity/v3/projects/25605943fb14406690b5924e55535b18 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:26.033758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac4efb-34a1-4aee-ae35-d8bb6eaba17e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:26.034042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac4efb-34a1-4aee-ae35-d8bb6eaba17e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:26.039167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac4efb-34a1-4aee-ae35-d8bb6eaba17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:26.070305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ce6ea935-20f8-4a19-9dc9-f3f0fd1b8e05 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:26.071691 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ce6ea935-20f8-4a19-9dc9-f3f0fd1b8e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:26.158937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-21ac4efb-34a1-4aee-ae35-d8bb6eaba17e None None] MFA Rules not processed for user `734d44438bfb4a79a68aa6705e49b1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:26.165373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-21ac4efb-34a1-4aee-ae35-d8bb6eaba17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:26.218887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:26.233462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 885/1747] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:26.239145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ea935-20f8-4a19-9dc9-f3f0fd1b8e05 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:26.241222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ce6ea935-20f8-4a19-9dc9-f3f0fd1b8e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:26.330532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 863/1748] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:26.332259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684db88-8a58-4b10-9c8a-fa4084fdfd92 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:26.332695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684db88-8a58-4b10-9c8a-fa4084fdfd92 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:26.332864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684db88-8a58-4b10-9c8a-fa4084fdfd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:26.369708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:26.378317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-4684db88-8a58-4b10-9c8a-fa4084fdfd92 None None] MFA Rules not processed for user `fb770d4379e74bf79ad4280329c771c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:26.379987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4684db88-8a58-4b10-9c8a-fa4084fdfd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:26.387402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ea935-20f8-4a19-9dc9-f3f0fd1b8e05 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:26.397088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672edd0b-eea1-41f8-93bd-029d926f4740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:26.397732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672edd0b-eea1-41f8-93bd-029d926f4740 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:26.398015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672edd0b-eea1-41f8-93bd-029d926f4740 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:26.398289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672edd0b-eea1-41f8-93bd-029d926f4740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:26.404868 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-672edd0b-eea1-41f8-93bd-029d926f4740 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:26.480689 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 864/1749] 10.222.0.58 () {58 vars in 1058 bytes} [Mon Aug 29 23:51:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:26.482091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af36d92-d456-4c63-b586-0174ecdb5a9e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:26.483364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af36d92-d456-4c63-b586-0174ecdb5a9e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:26.483735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af36d92-d456-4c63-b586-0174ecdb5a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:26.505178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-8af36d92-d456-4c63-b586-0174ecdb5a9e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:26.508100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8af36d92-d456-4c63-b586-0174ecdb5a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:26.523671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 865/1750] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:26.524608 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af36d92-d456-4c63-b586-0174ecdb5a9e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:26.545131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af36d92-d456-4c63-b586-0174ecdb5a9e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:26.553044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d7945-a1d8-4d06-8fe4-5e77aa871016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:26.553677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d7945-a1d8-4d06-8fe4-5e77aa871016 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:26.554002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d7945-a1d8-4d06-8fe4-5e77aa871016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:26.558398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d7945-a1d8-4d06-8fe4-5e77aa871016 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:26.566062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d7945-a1d8-4d06-8fe4-5e77aa871016 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:51:26.566062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d7945-a1d8-4d06-8fe4-5e77aa871016 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:26.566323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672edd0b-eea1-41f8-93bd-029d926f4740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:26.566453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d7945-a1d8-4d06-8fe4-5e77aa871016 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:26.566542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672edd0b-eea1-41f8-93bd-029d926f4740 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:26.588171 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 886/1751] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:26.589915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672edd0b-eea1-41f8-93bd-029d926f4740 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:26.601935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 866/1752] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:51:26.613849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672edd0b-eea1-41f8-93bd-029d926f4740 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:26.614788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a261b-1e3e-49bf-9b85-79c72f1cc6a2 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:26.615139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a261b-1e3e-49bf-9b85-79c72f1cc6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:26.615424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a261b-1e3e-49bf-9b85-79c72f1cc6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:26.621821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e911b-cbad-4e49-804f-9b2037394b30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:26.624202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e911b-cbad-4e49-804f-9b2037394b30 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:26.626721 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e911b-cbad-4e49-804f-9b2037394b30 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:26.626721 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e911b-cbad-4e49-804f-9b2037394b30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:26.630350 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-105e911b-cbad-4e49-804f-9b2037394b30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:26.662735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e911b-cbad-4e49-804f-9b2037394b30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:26.665053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e911b-cbad-4e49-804f-9b2037394b30 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:26.667673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f03a261b-1e3e-49bf-9b85-79c72f1cc6a2 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:26.668947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f03a261b-1e3e-49bf-9b85-79c72f1cc6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:26.695643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 867/1753] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:26.695643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a261b-1e3e-49bf-9b85-79c72f1cc6a2 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:26.703101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a261b-1e3e-49bf-9b85-79c72f1cc6a2 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:26.704536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 887/1754] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:26.705545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e911b-cbad-4e49-804f-9b2037394b30 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:26.708560 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf33f4e-e0a1-4e21-bae4-dc8532c37640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:26.709198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf33f4e-e0a1-4e21-bae4-dc8532c37640 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:26.709478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf33f4e-e0a1-4e21-bae4-dc8532c37640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:26.709753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf33f4e-e0a1-4e21-bae4-dc8532c37640 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:26.710405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf33f4e-e0a1-4e21-bae4-dc8532c37640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:26.712457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf33f4e-e0a1-4e21-bae4-dc8532c37640 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:26.770148 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e911b-cbad-4e49-804f-9b2037394b30 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:26.782045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e165a0-59eb-4ef3-83ab-6af0f06f6235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:26.788497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e165a0-59eb-4ef3-83ab-6af0f06f6235 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:26.788497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e165a0-59eb-4ef3-83ab-6af0f06f6235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:26.788497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e165a0-59eb-4ef3-83ab-6af0f06f6235 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:26.791381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e165a0-59eb-4ef3-83ab-6af0f06f6235 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:51:26.791979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e165a0-59eb-4ef3-83ab-6af0f06f6235 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:26.793788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e165a0-59eb-4ef3-83ab-6af0f06f6235 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:26.829973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 868/1755] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:26 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:26.839517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf33f4e-e0a1-4e21-bae4-dc8532c37640 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:26.858772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 888/1756] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:51:26.865127 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e165a0-59eb-4ef3-83ab-6af0f06f6235 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:26.898620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e165a0-59eb-4ef3-83ab-6af0f06f6235 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:27.083517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:27.123377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e83003-a798-4b55-b221-6e2506b87e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:27.126816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e83003-a798-4b55-b221-6e2506b87e5a admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:27.131578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e83003-a798-4b55-b221-6e2506b87e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:27.131869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e83003-a798-4b55-b221-6e2506b87e5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:27.132578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e83003-a798-4b55-b221-6e2506b87e5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:27.134891 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e83003-a798-4b55-b221-6e2506b87e5a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:27.186696 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf33f4e-e0a1-4e21-bae4-dc8532c37640 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:27.193525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596f884-713b-4980-b77b-a4a458f9ad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:27.194685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596f884-713b-4980-b77b-a4a458f9ad4f admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:27.195114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596f884-713b-4980-b77b-a4a458f9ad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:27.195558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596f884-713b-4980-b77b-a4a458f9ad4f admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:27.196368 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596f884-713b-4980-b77b-a4a458f9ad4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:27.200122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596f884-713b-4980-b77b-a4a458f9ad4f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:27.289088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 869/1757] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:26 2022] POST /identity/v3/users => generated 293 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:27.293625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596f884-713b-4980-b77b-a4a458f9ad4f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:27.312948 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596f884-713b-4980-b77b-a4a458f9ad4f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:27.332691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d826f1-4324-4432-8693-b1674011a289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:27.333401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d826f1-4324-4432-8693-b1674011a289 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:27.333595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d826f1-4324-4432-8693-b1674011a289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:27.333866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d826f1-4324-4432-8693-b1674011a289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:27.334571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d826f1-4324-4432-8693-b1674011a289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:27.336283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d826f1-4324-4432-8693-b1674011a289 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:27.353191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 889/1758] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:26 2022] POST /identity/v3/projects => generated 389 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:27.354710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e83003-a798-4b55-b221-6e2506b87e5a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:27.362233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 870/1759] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:51:27.378591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d826f1-4324-4432-8693-b1674011a289 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:27.400551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d826f1-4324-4432-8693-b1674011a289 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:27.592654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08830a3-f8bc-4887-af5c-ac223deb7e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:27.593389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08830a3-f8bc-4887-af5c-ac223deb7e18 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:27.593622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08830a3-f8bc-4887-af5c-ac223deb7e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:27.593883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08830a3-f8bc-4887-af5c-ac223deb7e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:27.594598 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08830a3-f8bc-4887-af5c-ac223deb7e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:27.596493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08830a3-f8bc-4887-af5c-ac223deb7e18 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:27.627816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e83003-a798-4b55-b221-6e2506b87e5a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:27.633159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 871/1760] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:27.634725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08830a3-f8bc-4887-af5c-ac223deb7e18 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:27.637599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf6a42-f144-4190-880f-65860d532c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:27.638364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf6a42-f144-4190-880f-65860d532c51 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:27.638752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf6a42-f144-4190-880f-65860d532c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:27.639101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf6a42-f144-4190-880f-65860d532c51 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:27.640333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf6a42-f144-4190-880f-65860d532c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:27.645147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf6a42-f144-4190-880f-65860d532c51 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:27.665130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:27.704263 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08830a3-f8bc-4887-af5c-ac223deb7e18 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:27.761306 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 890/1761] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:27 2022] POST /identity/v3/users => generated 295 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:27.766124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf6a42-f144-4190-880f-65860d532c51 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:27.770273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e145e2-0b5c-4b02-a933-f7d6fe235d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:27.779582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e145e2-0b5c-4b02-a933-f7d6fe235d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:27.779804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e145e2-0b5c-4b02-a933-f7d6fe235d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:27.780097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e145e2-0b5c-4b02-a933-f7d6fe235d90 admin admin] PATH_INFO: `/v3/projects/f95577114e5e44dc990567c47ed5f838/users/14cdcf58839b4a31b0cc969aacfe9b0f/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:27.827753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf6a42-f144-4190-880f-65860d532c51 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:27.840629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e145e2-0b5c-4b02-a933-f7d6fe235d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f95577114e5e44dc990567c47ed5f838, user_id=14cdcf58839b4a31b0cc969aacfe9b0f, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:27.842000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4d019-d87e-491e-bdb1-1ac2b1931e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:27.846622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4d019-d87e-491e-bdb1-1ac2b1931e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:27.848932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4d019-d87e-491e-bdb1-1ac2b1931e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:27.848932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4d019-d87e-491e-bdb1-1ac2b1931e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:27.849176 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e145e2-0b5c-4b02-a933-f7d6fe235d90 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:27.850082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4d019-d87e-491e-bdb1-1ac2b1931e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:27.858884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4d019-d87e-491e-bdb1-1ac2b1931e80 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:27.887214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 891/1762] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:27.899279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4d019-d87e-491e-bdb1-1ac2b1931e80 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:27.952221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4d019-d87e-491e-bdb1-1ac2b1931e80 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:27.964302 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659efbc1-5b2f-4e3c-9d35-80abe0084e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:27.965322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659efbc1-5b2f-4e3c-9d35-80abe0084e3f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:27.966997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659efbc1-5b2f-4e3c-9d35-80abe0084e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:27.967962 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659efbc1-5b2f-4e3c-9d35-80abe0084e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:27.969599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659efbc1-5b2f-4e3c-9d35-80abe0084e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:27.971535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 872/1763] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:27 2022] PUT /identity/v3/projects/f95577114e5e44dc990567c47ed5f838/users/14cdcf58839b4a31b0cc969aacfe9b0f/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:27.983040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659efbc1-5b2f-4e3c-9d35-80abe0084e3f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:27.987436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db489d7-a3b9-453b-9e8d-fbc704381009 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:27.992698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db489d7-a3b9-453b-9e8d-fbc704381009 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:27.992698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db489d7-a3b9-453b-9e8d-fbc704381009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:28.016427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 892/1764] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:28.018862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659efbc1-5b2f-4e3c-9d35-80abe0084e3f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:28.054768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659efbc1-5b2f-4e3c-9d35-80abe0084e3f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:28.179409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-1db489d7-a3b9-453b-9e8d-fbc704381009 None None] MFA Rules not processed for user `14cdcf58839b4a31b0cc969aacfe9b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:28.190252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1db489d7-a3b9-453b-9e8d-fbc704381009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:28.340722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf72d6f-b51d-48c5-a5f7-68b1adb563a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:28.349464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf72d6f-b51d-48c5-a5f7-68b1adb563a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:28.349464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf72d6f-b51d-48c5-a5f7-68b1adb563a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:28.349464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf72d6f-b51d-48c5-a5f7-68b1adb563a4 admin admin] PATH_INFO: `/v3/projects/00c68b02bedb4fac98254806539d48da/users/5f191d2624f549c9ae4d936a9f02bebf/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:28.375502 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 873/1765] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:28.376686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db489d7-a3b9-453b-9e8d-fbc704381009 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:28.394801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf72d6f-b51d-48c5-a5f7-68b1adb563a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00c68b02bedb4fac98254806539d48da, user_id=5f191d2624f549c9ae4d936a9f02bebf, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:28.397860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db489d7-a3b9-453b-9e8d-fbc704381009 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:28.403305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf72d6f-b51d-48c5-a5f7-68b1adb563a4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:28.426866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ad6ec-a053-4eb4-a105-907557fcd290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:28.434733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ad6ec-a053-4eb4-a105-907557fcd290 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:28.436317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ad6ec-a053-4eb4-a105-907557fcd290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:28.436760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ad6ec-a053-4eb4-a105-907557fcd290 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:28.440127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ad6ec-a053-4eb4-a105-907557fcd290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:28.453385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ad6ec-a053-4eb4-a105-907557fcd290 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:28.485004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:28.521223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 893/1766] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:28 2022] PUT /identity/v3/projects/00c68b02bedb4fac98254806539d48da/users/5f191d2624f549c9ae4d936a9f02bebf/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:28.555582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1e391-5572-425f-9616-8c83c6f7cc3e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:28.556452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1e391-5572-425f-9616-8c83c6f7cc3e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:28.559841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1e391-5572-425f-9616-8c83c6f7cc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:28.634908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 874/1767] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:28 2022] POST /identity/v3/projects => generated 387 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:28.636784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ad6ec-a053-4eb4-a105-907557fcd290 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:28.810129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-90e1e391-5572-425f-9616-8c83c6f7cc3e None None] MFA Rules not processed for user `5f191d2624f549c9ae4d936a9f02bebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:28.816594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ad6ec-a053-4eb4-a105-907557fcd290 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:28.821383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-90e1e391-5572-425f-9616-8c83c6f7cc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:28.849913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01bb9f-0949-4b74-98ec-1612856a3007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:28.851345 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01bb9f-0949-4b74-98ec-1612856a3007 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:28.852813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01bb9f-0949-4b74-98ec-1612856a3007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:28.853283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01bb9f-0949-4b74-98ec-1612856a3007 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:28.854134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01bb9f-0949-4b74-98ec-1612856a3007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:28.857609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01bb9f-0949-4b74-98ec-1612856a3007 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:29.026000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 894/1768] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:29.044510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 875/1769] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:28 2022] POST /identity/v3/users => generated 294 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:29.044782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1e391-5572-425f-9616-8c83c6f7cc3e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:29.044854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01bb9f-0949-4b74-98ec-1612856a3007 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:29.071436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1e391-5572-425f-9616-8c83c6f7cc3e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:29.084433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf9308-1140-4234-9853-3b98b2c7c0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:29.087858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf9308-1140-4234-9853-3b98b2c7c0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:29.088178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf9308-1140-4234-9853-3b98b2c7c0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:29.089371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf9308-1140-4234-9853-3b98b2c7c0e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:29.090800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf9308-1140-4234-9853-3b98b2c7c0e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:29.094155 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf9308-1140-4234-9853-3b98b2c7c0e2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:29.106845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01bb9f-0949-4b74-98ec-1612856a3007 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:29.120713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4da0d-c835-4525-9b84-58d6518fd864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:29.123371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4da0d-c835-4525-9b84-58d6518fd864 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:29.125095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4da0d-c835-4525-9b84-58d6518fd864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:29.125532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4da0d-c835-4525-9b84-58d6518fd864 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:29.128322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4da0d-c835-4525-9b84-58d6518fd864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:29.130679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4da0d-c835-4525-9b84-58d6518fd864 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:29.187233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 876/1770] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:29.202864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4da0d-c835-4525-9b84-58d6518fd864 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:29.225933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4da0d-c835-4525-9b84-58d6518fd864 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:29.242795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf209045-91c2-4aeb-8516-8ede3fb12271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:29.244903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209045-91c2-4aeb-8516-8ede3fb12271 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:29.245249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209045-91c2-4aeb-8516-8ede3fb12271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:29.245525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209045-91c2-4aeb-8516-8ede3fb12271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:29.246211 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf209045-91c2-4aeb-8516-8ede3fb12271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:29.247937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf209045-91c2-4aeb-8516-8ede3fb12271 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:29.253733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 895/1771] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:29 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:29.257042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf9308-1140-4234-9853-3b98b2c7c0e2 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:29.278318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 877/1772] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:29.290299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf209045-91c2-4aeb-8516-8ede3fb12271 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:29.320821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:29.323624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf209045-91c2-4aeb-8516-8ede3fb12271 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:29.464307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf9308-1140-4234-9853-3b98b2c7c0e2 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:29.495637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f8d59-c508-4ce9-ba91-582bcbb6769d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:29.496551 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f8d59-c508-4ce9-ba91-582bcbb6769d admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:29.497009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f8d59-c508-4ce9-ba91-582bcbb6769d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:29.497427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f8d59-c508-4ce9-ba91-582bcbb6769d admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:29.498218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f8d59-c508-4ce9-ba91-582bcbb6769d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:29.502013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f8d59-c508-4ce9-ba91-582bcbb6769d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:29.543059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb09eee-ccf8-4029-b068-6366edb41756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:29.544068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09eee-ccf8-4029-b068-6366edb41756 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:29.544586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09eee-ccf8-4029-b068-6366edb41756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:29.545018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09eee-ccf8-4029-b068-6366edb41756 admin admin] PATH_INFO: `/v3/projects/1c9ea9072e574b2f9604795ed49f3de7/users/6e9d3586f2b54423bb90091712975208/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:29.570791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb09eee-ccf8-4029-b068-6366edb41756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9ea9072e574b2f9604795ed49f3de7, user_id=6e9d3586f2b54423bb90091712975208, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:29.574938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb09eee-ccf8-4029-b068-6366edb41756 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:29.591084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 896/1773] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:29 2022] POST /identity/v3/users => generated 299 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:29.608112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f8d59-c508-4ce9-ba91-582bcbb6769d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:29.638751 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f8d59-c508-4ce9-ba91-582bcbb6769d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:29.648099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 878/1774] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:29 2022] PUT /identity/v3/projects/1c9ea9072e574b2f9604795ed49f3de7/users/6e9d3586f2b54423bb90091712975208/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:29.664690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af15c5-822d-4c3c-8af2-96f9a9723708 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:29.665021 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af15c5-822d-4c3c-8af2-96f9a9723708 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:29.665281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af15c5-822d-4c3c-8af2-96f9a9723708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:29.754490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-b4af15c5-822d-4c3c-8af2-96f9a9723708 None None] MFA Rules not processed for user `6e9d3586f2b54423bb90091712975208`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:29.759357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b4af15c5-822d-4c3c-8af2-96f9a9723708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:29.830662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:29.920473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 879/1775] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:29.921619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af15c5-822d-4c3c-8af2-96f9a9723708 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:29.940860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef2894-3c6d-4ad1-9355-d909a5d8438a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:29.941625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2894-3c6d-4ad1-9355-d909a5d8438a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:29.955291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2894-3c6d-4ad1-9355-d909a5d8438a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:29.955291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2894-3c6d-4ad1-9355-d909a5d8438a admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:29.956043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef2894-3c6d-4ad1-9355-d909a5d8438a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:29.959300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef2894-3c6d-4ad1-9355-d909a5d8438a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:29.981545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 897/1776] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:51:29.985709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef2894-3c6d-4ad1-9355-d909a5d8438a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:30.003479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef2894-3c6d-4ad1-9355-d909a5d8438a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:30.011605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15141a03-9651-465f-a52f-9a6e1907b535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:30.012442 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141a03-9651-465f-a52f-9a6e1907b535 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:30.012608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141a03-9651-465f-a52f-9a6e1907b535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:30.012919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141a03-9651-465f-a52f-9a6e1907b535 admin admin] PATH_INFO: `/v3/projects/b31e7af6f0a447ad83d8f6f6cd16aef4/users/ee372a8f67c44d7686bf7f17a00f6f89/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:30.064359 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15141a03-9651-465f-a52f-9a6e1907b535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31e7af6f0a447ad83d8f6f6cd16aef4, user_id=ee372a8f67c44d7686bf7f17a00f6f89, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:30.071260 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15141a03-9651-465f-a52f-9a6e1907b535 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:30.077209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af15c5-822d-4c3c-8af2-96f9a9723708 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:30.086716 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ee172-d672-4954-8577-1e0a8e88dab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:30.087329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ee172-d672-4954-8577-1e0a8e88dab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:30.087606 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ee172-d672-4954-8577-1e0a8e88dab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:30.087937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ee172-d672-4954-8577-1e0a8e88dab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:30.088757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ee172-d672-4954-8577-1e0a8e88dab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:30.091763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ee172-d672-4954-8577-1e0a8e88dab7 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:30.193261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 898/1777] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:29 2022] PUT /identity/v3/projects/b31e7af6f0a447ad83d8f6f6cd16aef4/users/ee372a8f67c44d7686bf7f17a00f6f89/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:30.197146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15141a03-9651-465f-a52f-9a6e1907b535 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:30.246076 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 880/1778] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:29 2022] POST /identity/v3/projects => generated 387 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:30.250620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ee172-d672-4954-8577-1e0a8e88dab7 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:30.456364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ee172-d672-4954-8577-1e0a8e88dab7 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:30.477125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254f88b-5db7-46bb-ab2d-66560c58c768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:30.479692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254f88b-5db7-46bb-ab2d-66560c58c768 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:30.484240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254f88b-5db7-46bb-ab2d-66560c58c768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:30.484696 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254f88b-5db7-46bb-ab2d-66560c58c768 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:30.485411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254f88b-5db7-46bb-ab2d-66560c58c768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:30.491913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254f88b-5db7-46bb-ab2d-66560c58c768 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:30.545573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15141a03-9651-465f-a52f-9a6e1907b535 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:30.567365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcfe68-8621-4c27-89fd-c694c47dafcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:30.568479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcfe68-8621-4c27-89fd-c694c47dafcd admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:30.569070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcfe68-8621-4c27-89fd-c694c47dafcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:30.569658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcfe68-8621-4c27-89fd-c694c47dafcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:30.571208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcfe68-8621-4c27-89fd-c694c47dafcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:30.583093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcfe68-8621-4c27-89fd-c694c47dafcd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:30.621674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 881/1779] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:30 2022] POST /identity/v3/users => generated 300 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:30.626939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254f88b-5db7-46bb-ab2d-66560c58c768 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:30.630788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 899/1780] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:51:30.635630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcfe68-8621-4c27-89fd-c694c47dafcd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:30.651343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcfe68-8621-4c27-89fd-c694c47dafcd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:30.661472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50a4db-bc4a-44aa-8173-8ab823d2e7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:30.662488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50a4db-bc4a-44aa-8173-8ab823d2e7e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:30.662984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50a4db-bc4a-44aa-8173-8ab823d2e7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:30.663408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50a4db-bc4a-44aa-8173-8ab823d2e7e2 admin admin] PATH_INFO: `/v3/projects/b31e7af6f0a447ad83d8f6f6cd16aef4/users/ee372a8f67c44d7686bf7f17a00f6f89/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:30.663652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254f88b-5db7-46bb-ab2d-66560c58c768 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:30.666591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50a4db-bc4a-44aa-8173-8ab823d2e7e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31e7af6f0a447ad83d8f6f6cd16aef4, user_id=ee372a8f67c44d7686bf7f17a00f6f89, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:30.669915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50a4db-bc4a-44aa-8173-8ab823d2e7e2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:30.680415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185662d-56b6-46ac-b798-2177cc5ed797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:30.681443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185662d-56b6-46ac-b798-2177cc5ed797 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:30.681712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185662d-56b6-46ac-b798-2177cc5ed797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:30.681987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185662d-56b6-46ac-b798-2177cc5ed797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:30.682670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185662d-56b6-46ac-b798-2177cc5ed797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:30.684285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185662d-56b6-46ac-b798-2177cc5ed797 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:30.715020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 882/1781] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:30.716060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185662d-56b6-46ac-b798-2177cc5ed797 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:30.738279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 900/1782] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:30 2022] PUT /identity/v3/projects/b31e7af6f0a447ad83d8f6f6cd16aef4/users/ee372a8f67c44d7686bf7f17a00f6f89/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:30.740344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ebbc6-db90-486d-b5ac-25c5051686ec None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:30.740644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ebbc6-db90-486d-b5ac-25c5051686ec None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:30.740918 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ebbc6-db90-486d-b5ac-25c5051686ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:30.765460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185662d-56b6-46ac-b798-2177cc5ed797 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:30.789772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:30.835865 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:30.920042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-155ebbc6-db90-486d-b5ac-25c5051686ec None None] MFA Rules not processed for user `ee372a8f67c44d7686bf7f17a00f6f89`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:30.921020 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-155ebbc6-db90-486d-b5ac-25c5051686ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:30.931199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23eee99-b84c-4fcb-8f4d-c24de2e1ccbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:30.931889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eee99-b84c-4fcb-8f4d-c24de2e1ccbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:30.932171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eee99-b84c-4fcb-8f4d-c24de2e1ccbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:30.934147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eee99-b84c-4fcb-8f4d-c24de2e1ccbe admin admin] PATH_INFO: `/v3/projects/59bc3058364f4d418cd7757164f5e254/users/e18acfc590734fb8a76bb5584d443b4c/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:30.968464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23eee99-b84c-4fcb-8f4d-c24de2e1ccbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bc3058364f4d418cd7757164f5e254, user_id=e18acfc590734fb8a76bb5584d443b4c, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:30.971631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23eee99-b84c-4fcb-8f4d-c24de2e1ccbe admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:31.101194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 883/1783] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:30 2022] PUT /identity/v3/projects/59bc3058364f4d418cd7757164f5e254/users/e18acfc590734fb8a76bb5584d443b4c/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:31.107236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23eee99-b84c-4fcb-8f4d-c24de2e1ccbe admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:31.172238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 901/1784] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:31.194537 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ebbc6-db90-486d-b5ac-25c5051686ec None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:31.196572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-155ebbc6-db90-486d-b5ac-25c5051686ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:31.304894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23eee99-b84c-4fcb-8f4d-c24de2e1ccbe admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:31.326086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8399123-8a1e-418b-9790-87bf6b0a25a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:31.326574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8399123-8a1e-418b-9790-87bf6b0a25a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:31.326854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8399123-8a1e-418b-9790-87bf6b0a25a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:31.327130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8399123-8a1e-418b-9790-87bf6b0a25a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:31.328169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8399123-8a1e-418b-9790-87bf6b0a25a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:31.329873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8399123-8a1e-418b-9790-87bf6b0a25a7 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:31.347632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ebbc6-db90-486d-b5ac-25c5051686ec None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:31.372932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 884/1785] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:31.374398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9589c86-145a-4ffe-bdac-65c980a3d9ae None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:31.374729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9589c86-145a-4ffe-bdac-65c980a3d9ae None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:31.375005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9589c86-145a-4ffe-bdac-65c980a3d9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:31.444348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d9589c86-145a-4ffe-bdac-65c980a3d9ae None None] MFA Rules not processed for user `34ef7ee27c3948ac9df1ff94ce24618e`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:31.447608 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d9589c86-145a-4ffe-bdac-65c980a3d9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:31.449151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e95d3-91e2-45cc-8ce4-ece2b909c09c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:31.450115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e95d3-91e2-45cc-8ce4-ece2b909c09c service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:31.450592 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e95d3-91e2-45cc-8ce4-ece2b909c09c service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:31.451089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e95d3-91e2-45cc-8ce4-ece2b909c09c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:31.586220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 885/1786] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:31.591308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9589c86-145a-4ffe-bdac-65c980a3d9ae None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:31.655056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9589c86-145a-4ffe-bdac-65c980a3d9ae None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:31.676793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:31.681848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc3b28-55d0-4ad9-a49b-9a4633e2bc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:31.682506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e95d3-91e2-45cc-8ce4-ece2b909c09c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:31.682636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc3b28-55d0-4ad9-a49b-9a4633e2bc83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:31.682887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc3b28-55d0-4ad9-a49b-9a4633e2bc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:31.683172 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc3b28-55d0-4ad9-a49b-9a4633e2bc83 admin admin] PATH_INFO: `/v3/projects/59bc3058364f4d418cd7757164f5e254/users/e18acfc590734fb8a76bb5584d443b4c/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:31.683773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e95d3-91e2-45cc-8ce4-ece2b909c09c service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:31.691707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc3b28-55d0-4ad9-a49b-9a4633e2bc83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bc3058364f4d418cd7757164f5e254, user_id=e18acfc590734fb8a76bb5584d443b4c, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:31.694758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc3b28-55d0-4ad9-a49b-9a4633e2bc83 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:31.726152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 886/1787] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:31 2022] PUT /identity/v3/projects/59bc3058364f4d418cd7757164f5e254/users/e18acfc590734fb8a76bb5584d443b4c/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:31.726490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 902/1788] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:51:31.730312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5373c-7e68-4a8a-8ec9-6b6f6632772b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:31.730900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5373c-7e68-4a8a-8ec9-6b6f6632772b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:31.731321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5373c-7e68-4a8a-8ec9-6b6f6632772b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:31.811638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-77c5373c-7e68-4a8a-8ec9-6b6f6632772b None None] MFA Rules not processed for user `e18acfc590734fb8a76bb5584d443b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:31.813072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-77c5373c-7e68-4a8a-8ec9-6b6f6632772b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:31.874593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:31.982456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 887/1789] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:31.994980 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5373c-7e68-4a8a-8ec9-6b6f6632772b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:31.996090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e95d3-91e2-45cc-8ce4-ece2b909c09c service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:32.022947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e95d3-91e2-45cc-8ce4-ece2b909c09c service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:32.133413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5373c-7e68-4a8a-8ec9-6b6f6632772b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:32.137353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d4d84-aa63-4d29-ba17-582f0ec7af66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:32.138159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d4d84-aa63-4d29-ba17-582f0ec7af66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:32.138897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821dd0d-8b7b-4236-9527-3da9a4043c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:32.139037 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d4d84-aa63-4d29-ba17-582f0ec7af66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:32.139432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d4d84-aa63-4d29-ba17-582f0ec7af66 admin admin] PATH_INFO: `/v3/users/5f191d2624f549c9ae4d936a9f02bebf` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:32.139977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821dd0d-8b7b-4236-9527-3da9a4043c74 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:32.140478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821dd0d-8b7b-4236-9527-3da9a4043c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:32.140851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821dd0d-8b7b-4236-9527-3da9a4043c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:32.141112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d4d84-aa63-4d29-ba17-582f0ec7af66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f191d2624f549c9ae4d936a9f02bebf)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:32.143513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d4d84-aa63-4d29-ba17-582f0ec7af66 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:32.159740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821dd0d-8b7b-4236-9527-3da9a4043c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:32.162646 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821dd0d-8b7b-4236-9527-3da9a4043c74 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:32.180234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 888/1790] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:32.366051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e18d4d84-aa63-4d29-ba17-582f0ec7af66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f191d2624f549c9ae4d936a9f02bebf'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:32.389035 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e18d4d84-aa63-4d29-ba17-582f0ec7af66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f191d2624f549c9ae4d936a9f02bebf'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:32.403237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e18d4d84-aa63-4d29-ba17-582f0ec7af66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f191d2624f549c9ae4d936a9f02bebf'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:32.527488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e18d4d84-aa63-4d29-ba17-582f0ec7af66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f191d2624f549c9ae4d936a9f02bebf'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:32.528618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e18d4d84-aa63-4d29-ba17-582f0ec7af66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f191d2624f549c9ae4d936a9f02bebf'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:32.561044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 903/1791] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:31 2022] DELETE /identity/v3/users/5f191d2624f549c9ae4d936a9f02bebf => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:32.571230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c301c6e-a9f8-4823-b1e9-581be3d73ff5 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:32.571865 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c301c6e-a9f8-4823-b1e9-581be3d73ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:32.574826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c301c6e-a9f8-4823-b1e9-581be3d73ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:32.620979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-6c301c6e-a9f8-4823-b1e9-581be3d73ff5 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:32.622237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6c301c6e-a9f8-4823-b1e9-581be3d73ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:32.752612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 889/1792] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:32.755081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa7230-6c5b-4b5e-a1dc-a84027ffbe34 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:32.755081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa7230-6c5b-4b5e-a1dc-a84027ffbe34 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:32.755314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa7230-6c5b-4b5e-a1dc-a84027ffbe34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:32.767328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c301c6e-a9f8-4823-b1e9-581be3d73ff5 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:32.773481 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6c301c6e-a9f8-4823-b1e9-581be3d73ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:32.809901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ecaa7230-6c5b-4b5e-a1dc-a84027ffbe34 None None] MFA Rules not processed for user `90b2c6a74c29443b94d1577b39c7ce0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:32.813318 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ecaa7230-6c5b-4b5e-a1dc-a84027ffbe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:32.901587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c301c6e-a9f8-4823-b1e9-581be3d73ff5 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:32.912326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6946d-3d45-410e-aef0-9e4e1097831b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:32.912326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6946d-3d45-410e-aef0-9e4e1097831b service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:32.912326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6946d-3d45-410e-aef0-9e4e1097831b service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:32.912326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6946d-3d45-410e-aef0-9e4e1097831b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:32.919840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6946d-3d45-410e-aef0-9e4e1097831b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:32.922575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6946d-3d45-410e-aef0-9e4e1097831b service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:32.950761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 904/1793] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:32.965831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 890/1794] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:51:34.117059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa7230-6c5b-4b5e-a1dc-a84027ffbe34 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:34.120183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ecaa7230-6c5b-4b5e-a1dc-a84027ffbe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:34.174297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa7230-6c5b-4b5e-a1dc-a84027ffbe34 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:34.190537 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:34.191857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:34.192203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:34.192538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] PATH_INFO: `/v3/projects/00c68b02bedb4fac98254806539d48da` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:34.195264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00c68b02bedb4fac98254806539d48da)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:34.197353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6946d-3d45-410e-aef0-9e4e1097831b service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:34.198857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5fe6946d-3d45-410e-aef0-9e4e1097831b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:34.200963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:34.230570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:34.265502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:34.379908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00c68b02bedb4fac98254806539d48da', 'request_id': 'req-644d12d8-ba62-45a7-b6b3-fcd485e70827'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:34.403264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6946d-3d45-410e-aef0-9e4e1097831b service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:34.404096 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00c68b02bedb4fac98254806539d48da', 'request_id': 'req-644d12d8-ba62-45a7-b6b3-fcd485e70827'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:34.591194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf213a8-1b39-4791-ba93-4dbe33fbf005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:34.594350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf213a8-1b39-4791-ba93-4dbe33fbf005 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:34.596088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf213a8-1b39-4791-ba93-4dbe33fbf005 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:34.600023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf213a8-1b39-4791-ba93-4dbe33fbf005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:34.607180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00c68b02bedb4fac98254806539d48da', 'request_id': 'req-644d12d8-ba62-45a7-b6b3-fcd485e70827'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:34.623011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3cf213a8-1b39-4791-ba93-4dbe33fbf005 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:34.639178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] The token cache is being invalidate because project 00c68b02bedb4fac98254806539d48da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:34.639178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:34.641795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:34.651396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 905/1795] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:34 2022] DELETE /identity/v3/projects/00c68b02bedb4fac98254806539d48da => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:34.655329 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:34.668025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:34.850288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf213a8-1b39-4791-ba93-4dbe33fbf005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:34.851789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf213a8-1b39-4791-ba93-4dbe33fbf005 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:34.869066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 891/1796] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:51:34.874303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d12d8-ba62-45a7-b6b3-fcd485e70827 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:34.883202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:34.885809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:34.886717 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:34.887077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] PATH_INFO: `/v3/users/ee372a8f67c44d7686bf7f17a00f6f89` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:34.888520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee372a8f67c44d7686bf7f17a00f6f89)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:34.891946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:35.197489 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee372a8f67c44d7686bf7f17a00f6f89'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:35.337586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee372a8f67c44d7686bf7f17a00f6f89'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:35.368238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee372a8f67c44d7686bf7f17a00f6f89'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:35.458597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee372a8f67c44d7686bf7f17a00f6f89'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:35.459459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee372a8f67c44d7686bf7f17a00f6f89'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:35.471198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 906/1797] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:34 2022] DELETE /identity/v3/users/ee372a8f67c44d7686bf7f17a00f6f89 => generated 0 bytes in 814 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:35.845875 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59177513-c6c9-4fb9-8653-feee299b394c None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:35.846803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59177513-c6c9-4fb9-8653-feee299b394c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:35.847482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59177513-c6c9-4fb9-8653-feee299b394c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:36.001415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-59177513-c6c9-4fb9-8653-feee299b394c None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:36.002695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-59177513-c6c9-4fb9-8653-feee299b394c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:36.239117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:36.276046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 892/1798] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:51:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:36.703099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:36.704621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:36.773970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:36.997427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a25a96-29e4-4970-b65c-cf34c4ca389c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:37.013666 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86475f8d-9107-4b24-8383-5661e6e046a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:37.014923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86475f8d-9107-4b24-8383-5661e6e046a8 service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:37.015354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86475f8d-9107-4b24-8383-5661e6e046a8 service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:37.015953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86475f8d-9107-4b24-8383-5661e6e046a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:37.041435 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86475f8d-9107-4b24-8383-5661e6e046a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:37.043625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86475f8d-9107-4b24-8383-5661e6e046a8 service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:37.075486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 907/1799] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:51:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:51:38.147916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59177513-c6c9-4fb9-8653-feee299b394c None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:38.150946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-59177513-c6c9-4fb9-8653-feee299b394c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:38.325762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59177513-c6c9-4fb9-8653-feee299b394c None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:38.357785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:38.362470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:38.363712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:38.364187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] PATH_INFO: `/v3/projects/b31e7af6f0a447ad83d8f6f6cd16aef4` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:38.369743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31e7af6f0a447ad83d8f6f6cd16aef4)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:38.376340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:38.583803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31e7af6f0a447ad83d8f6f6cd16aef4', 'request_id': 'req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:38.724535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31e7af6f0a447ad83d8f6f6cd16aef4', 'request_id': 'req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:38.732471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31e7af6f0a447ad83d8f6f6cd16aef4', 'request_id': 'req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:38.760705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] The token cache is being invalidate because project b31e7af6f0a447ad83d8f6f6cd16aef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:38.761597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:38.763290 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-26a9b781-5a9d-4c32-8e17-b6a4dd3d02af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:38.777613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 893/1800] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:38 2022] DELETE /identity/v3/projects/b31e7af6f0a447ad83d8f6f6cd16aef4 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:38.778599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ed005-d212-4a90-b3fa-2aa37a413f7d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:38.779339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ed005-d212-4a90-b3fa-2aa37a413f7d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:38.779339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ed005-d212-4a90-b3fa-2aa37a413f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:38.824986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-309ed005-d212-4a90-b3fa-2aa37a413f7d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:38.826243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-309ed005-d212-4a90-b3fa-2aa37a413f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:38.927511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 908/1801] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:38.935865 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40451cd-e4a7-4a96-8f93-cd09facd667f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:38.942228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40451cd-e4a7-4a96-8f93-cd09facd667f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:38.942228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40451cd-e4a7-4a96-8f93-cd09facd667f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:38.959901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-a40451cd-e4a7-4a96-8f93-cd09facd667f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:38.961147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a40451cd-e4a7-4a96-8f93-cd09facd667f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:38.976052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ed005-d212-4a90-b3fa-2aa37a413f7d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:38.983417 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 894/1802] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:38.988819 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ed005-d212-4a90-b3fa-2aa37a413f7d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:38.996225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1c846-4c2c-4e62-bd89-93a9e516e8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:38.998377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1c846-4c2c-4e62-bd89-93a9e516e8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:38.999427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1c846-4c2c-4e62-bd89-93a9e516e8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:39.000144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1c846-4c2c-4e62-bd89-93a9e516e8f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:39.001393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1c846-4c2c-4e62-bd89-93a9e516e8f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:51:39.001956 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1c846-4c2c-4e62-bd89-93a9e516e8f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:39.004848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1c846-4c2c-4e62-bd89-93a9e516e8f9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:39.025575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40451cd-e4a7-4a96-8f93-cd09facd667f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:39.032006 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 909/1803] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:51:39.037168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40451cd-e4a7-4a96-8f93-cd09facd667f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:39.044610 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70894adb-526c-4cbb-bcf6-000517bdca8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:39.045687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70894adb-526c-4cbb-bcf6-000517bdca8d admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:39.046181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70894adb-526c-4cbb-bcf6-000517bdca8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:39.046709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70894adb-526c-4cbb-bcf6-000517bdca8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:39.047556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70894adb-526c-4cbb-bcf6-000517bdca8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:39.050195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70894adb-526c-4cbb-bcf6-000517bdca8d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:39.216637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 895/1804] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:39 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:39.220992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1c846-4c2c-4e62-bd89-93a9e516e8f9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:39.232126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1c846-4c2c-4e62-bd89-93a9e516e8f9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:39.313221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb4411-b3d6-4a1c-91db-b3a9e204bf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:39.314799 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb4411-b3d6-4a1c-91db-b3a9e204bf5d admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:39.315281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb4411-b3d6-4a1c-91db-b3a9e204bf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:39.315966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb4411-b3d6-4a1c-91db-b3a9e204bf5d admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:39.317113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb4411-b3d6-4a1c-91db-b3a9e204bf5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:39.320766 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb4411-b3d6-4a1c-91db-b3a9e204bf5d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:39.419882 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 910/1805] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:39 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:51:39.424552 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70894adb-526c-4cbb-bcf6-000517bdca8d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:39.436793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70894adb-526c-4cbb-bcf6-000517bdca8d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:39.443720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e143b6-1d70-4b33-8413-9f187056e5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:39.445785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e143b6-1d70-4b33-8413-9f187056e5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:39.445785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e143b6-1d70-4b33-8413-9f187056e5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:39.445785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e143b6-1d70-4b33-8413-9f187056e5a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:39.446467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e143b6-1d70-4b33-8413-9f187056e5a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:39.448246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e143b6-1d70-4b33-8413-9f187056e5a5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:39.468270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 896/1806] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:39.469416 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb4411-b3d6-4a1c-91db-b3a9e204bf5d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:39.484620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb4411-b3d6-4a1c-91db-b3a9e204bf5d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:39.494325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef971077-5d84-437c-9deb-c69c717ac061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:39.498270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971077-5d84-437c-9deb-c69c717ac061 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:39.498270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971077-5d84-437c-9deb-c69c717ac061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:39.498270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971077-5d84-437c-9deb-c69c717ac061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:39.498270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef971077-5d84-437c-9deb-c69c717ac061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:39.498270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef971077-5d84-437c-9deb-c69c717ac061 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:39.501839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:39.512639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 911/1807] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:51:39.516630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e143b6-1d70-4b33-8413-9f187056e5a5 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:39.530359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e143b6-1d70-4b33-8413-9f187056e5a5 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:39.536521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043b2a7-73e4-41f6-957d-ede7cc753507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:39.537503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043b2a7-73e4-41f6-957d-ede7cc753507 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:39.537932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043b2a7-73e4-41f6-957d-ede7cc753507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:39.547394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043b2a7-73e4-41f6-957d-ede7cc753507 admin admin] PATH_INFO: `/v3/projects/297ff4b2a2ec44f0b017304e3f08822d/users/ebc9834edbe0457bb2f830c59d72c2e0/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:39.561615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043b2a7-73e4-41f6-957d-ede7cc753507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=297ff4b2a2ec44f0b017304e3f08822d, user_id=ebc9834edbe0457bb2f830c59d72c2e0, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:39.564430 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043b2a7-73e4-41f6-957d-ede7cc753507 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:39.670647 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 897/1808] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:39 2022] PUT /identity/v3/projects/297ff4b2a2ec44f0b017304e3f08822d/users/ebc9834edbe0457bb2f830c59d72c2e0/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:39.682930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d37ea-3f69-4e84-ab19-1a5ab13635bb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:39.683220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d37ea-3f69-4e84-ab19-1a5ab13635bb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:39.683492 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d37ea-3f69-4e84-ab19-1a5ab13635bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:39.716062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a22d37ea-3f69-4e84-ab19-1a5ab13635bb None None] MFA Rules not processed for user `ebc9834edbe0457bb2f830c59d72c2e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:39.717560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a22d37ea-3f69-4e84-ab19-1a5ab13635bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:39.810763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 912/1809] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:39.815522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043b2a7-73e4-41f6-957d-ede7cc753507 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:39.842490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:39.899301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043b2a7-73e4-41f6-957d-ede7cc753507 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:39.904655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b9226-4e73-4486-9698-50362a8f1e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:39.905407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b9226-4e73-4486-9698-50362a8f1e0a admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:39.905936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b9226-4e73-4486-9698-50362a8f1e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:39.906310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b9226-4e73-4486-9698-50362a8f1e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:39.907099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b9226-4e73-4486-9698-50362a8f1e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:39.910182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b9226-4e73-4486-9698-50362a8f1e0a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:39.986290 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 898/1810] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:39 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:39.991468 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d37ea-3f69-4e84-ab19-1a5ab13635bb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:40.003340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d37ea-3f69-4e84-ab19-1a5ab13635bb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:40.068336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b23f70-8c55-47c7-bd72-e4988ee40c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:40.068867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b23f70-8c55-47c7-bd72-e4988ee40c25 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:40.069146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b23f70-8c55-47c7-bd72-e4988ee40c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:40.069418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b23f70-8c55-47c7-bd72-e4988ee40c25 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:40.071142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b23f70-8c55-47c7-bd72-e4988ee40c25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:40.073628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b23f70-8c55-47c7-bd72-e4988ee40c25 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:40.176094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 913/1811] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:39 2022] POST /identity/v3/users => generated 300 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:40.178637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b9226-4e73-4486-9698-50362a8f1e0a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:40.195802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b9226-4e73-4486-9698-50362a8f1e0a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:40.203975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7dede-a71b-4a7f-b9c4-3c01d9fd8e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:40.206442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7dede-a71b-4a7f-b9c4-3c01d9fd8e18 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:40.207572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7dede-a71b-4a7f-b9c4-3c01d9fd8e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:40.208278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7dede-a71b-4a7f-b9c4-3c01d9fd8e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:40.208435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7dede-a71b-4a7f-b9c4-3c01d9fd8e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:40.210202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7dede-a71b-4a7f-b9c4-3c01d9fd8e18 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:40.232401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b23f70-8c55-47c7-bd72-e4988ee40c25 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:40.232806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 899/1812] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:40.243567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b23f70-8c55-47c7-bd72-e4988ee40c25 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:40.248465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b1bad-eabe-4d7d-8015-c3f2a13b11e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:40.249236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b1bad-eabe-4d7d-8015-c3f2a13b11e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:40.249424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b1bad-eabe-4d7d-8015-c3f2a13b11e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:40.249701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b1bad-eabe-4d7d-8015-c3f2a13b11e9 admin admin] PATH_INFO: `/v3/projects/565e20ccd0d145b6be944f10d58a9678/users/c3606cf6948f419eb31e22ab88a34e70/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:40.289239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b1bad-eabe-4d7d-8015-c3f2a13b11e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=565e20ccd0d145b6be944f10d58a9678, user_id=c3606cf6948f419eb31e22ab88a34e70, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:40.292135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b1bad-eabe-4d7d-8015-c3f2a13b11e9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:40.392424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 914/1813] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:40 2022] PUT /identity/v3/projects/565e20ccd0d145b6be944f10d58a9678/users/c3606cf6948f419eb31e22ab88a34e70/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:40.400486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7dede-a71b-4a7f-b9c4-3c01d9fd8e18 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:40.413239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7dede-a71b-4a7f-b9c4-3c01d9fd8e18 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:40.497508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803e7d4-7487-4c76-a006-30a9179104cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:40.498132 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803e7d4-7487-4c76-a006-30a9179104cc admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:40.498415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803e7d4-7487-4c76-a006-30a9179104cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:40.498721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803e7d4-7487-4c76-a006-30a9179104cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:40.499518 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803e7d4-7487-4c76-a006-30a9179104cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:40.501133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803e7d4-7487-4c76-a006-30a9179104cc admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:40.531008 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 900/1814] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:40.531520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b1bad-eabe-4d7d-8015-c3f2a13b11e9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:40.554719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b1bad-eabe-4d7d-8015-c3f2a13b11e9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:40.560201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880693c-1d3c-418e-835f-12bd9212a390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:40.561308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880693c-1d3c-418e-835f-12bd9212a390 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:40.561308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880693c-1d3c-418e-835f-12bd9212a390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:40.562071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880693c-1d3c-418e-835f-12bd9212a390 admin admin] PATH_INFO: `/v3/projects/565e20ccd0d145b6be944f10d58a9678/users/c3606cf6948f419eb31e22ab88a34e70/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:40.564285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880693c-1d3c-418e-835f-12bd9212a390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=565e20ccd0d145b6be944f10d58a9678, user_id=c3606cf6948f419eb31e22ab88a34e70, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:40.568307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880693c-1d3c-418e-835f-12bd9212a390 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:40.587756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 915/1815] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:40 2022] PUT /identity/v3/projects/565e20ccd0d145b6be944f10d58a9678/users/c3606cf6948f419eb31e22ab88a34e70/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:40.600691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3ad95-3032-42d4-9cca-52a7ebcf39ef None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:40.601007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3ad95-3032-42d4-9cca-52a7ebcf39ef None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:40.601284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3ad95-3032-42d4-9cca-52a7ebcf39ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:40.656860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-80a3ad95-3032-42d4-9cca-52a7ebcf39ef None None] MFA Rules not processed for user `c3606cf6948f419eb31e22ab88a34e70`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:40.658539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-80a3ad95-3032-42d4-9cca-52a7ebcf39ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:40.775697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 901/1816] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:40.786303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880693c-1d3c-418e-835f-12bd9212a390 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:40.788181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8880693c-1d3c-418e-835f-12bd9212a390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:40.893398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:40.913059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880693c-1d3c-418e-835f-12bd9212a390 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:40.918611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6540f11-a453-4a91-a06f-87af54899b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:40.919519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6540f11-a453-4a91-a06f-87af54899b13 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:40.919895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6540f11-a453-4a91-a06f-87af54899b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:40.920258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6540f11-a453-4a91-a06f-87af54899b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:40.932647 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6540f11-a453-4a91-a06f-87af54899b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:40.934561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6540f11-a453-4a91-a06f-87af54899b13 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:40.958979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 916/1817] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:42.648205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3ad95-3032-42d4-9cca-52a7ebcf39ef None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:42.682198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:42.763186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3ad95-3032-42d4-9cca-52a7ebcf39ef None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:42.768114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:42.772092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:42.772092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:42.772092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] PATH_INFO: `/v3/users/ebc9834edbe0457bb2f830c59d72c2e0` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:42.772092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebc9834edbe0457bb2f830c59d72c2e0)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:42.773273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:43.091067 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebc9834edbe0457bb2f830c59d72c2e0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:43.119332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebc9834edbe0457bb2f830c59d72c2e0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:43.138867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebc9834edbe0457bb2f830c59d72c2e0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:43.230723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebc9834edbe0457bb2f830c59d72c2e0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:43.231662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebc9834edbe0457bb2f830c59d72c2e0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:43.240686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 902/1818] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:42 2022] DELETE /identity/v3/users/ebc9834edbe0457bb2f830c59d72c2e0 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:43.246114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85c11e-bf1b-4441-8f41-c0bb9c131a30 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:43.246883 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85c11e-bf1b-4441-8f41-c0bb9c131a30 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:43.246883 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85c11e-bf1b-4441-8f41-c0bb9c131a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:43.273512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-fa85c11e-bf1b-4441-8f41-c0bb9c131a30 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:43.275207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fa85c11e-bf1b-4441-8f41-c0bb9c131a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:43.400236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 917/1819] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:43.411857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:43.413299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:43.493919 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358ebde-c1f4-4522-907d-676e051b78c4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:43.498258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eb8fe-c79a-4f04-a070-36b405e254a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:43.521488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eb8fe-c79a-4f04-a070-36b405e254a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:43.522132 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eb8fe-c79a-4f04-a070-36b405e254a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:43.525124 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eb8fe-c79a-4f04-a070-36b405e254a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:43.546459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223eb8fe-c79a-4f04-a070-36b405e254a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:43.549112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223eb8fe-c79a-4f04-a070-36b405e254a0 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:43.569661 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 903/1820] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:44.283500 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85c11e-bf1b-4441-8f41-c0bb9c131a30 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:44.290042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fa85c11e-bf1b-4441-8f41-c0bb9c131a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:44.348153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85c11e-bf1b-4441-8f41-c0bb9c131a30 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:44.359228 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:44.360154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:44.360433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:44.360878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] PATH_INFO: `/v3/projects/297ff4b2a2ec44f0b017304e3f08822d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:44.362844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=297ff4b2a2ec44f0b017304e3f08822d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:44.366033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:44.516018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eb8fe-c79a-4f04-a070-36b405e254a0 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:44.518194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-223eb8fe-c79a-4f04-a070-36b405e254a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:44.567284 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297ff4b2a2ec44f0b017304e3f08822d', 'request_id': 'req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:44.577110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297ff4b2a2ec44f0b017304e3f08822d', 'request_id': 'req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:44.610238 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:44.642887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eb8fe-c79a-4f04-a070-36b405e254a0 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:44.675812 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297ff4b2a2ec44f0b017304e3f08822d', 'request_id': 'req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:44.713488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfba70-3dbd-42cc-abec-abd0491b4cd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:44.714256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfba70-3dbd-42cc-abec-abd0491b4cd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:44.721526 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfba70-3dbd-42cc-abec-abd0491b4cd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:44.721526 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfba70-3dbd-42cc-abec-abd0491b4cd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:44.721715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] The token cache is being invalidate because project 297ff4b2a2ec44f0b017304e3f08822d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:44.721715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:44.721715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:44.721927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d8bfba70-3dbd-42cc-abec-abd0491b4cd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:44.722002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 918/1821] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:44 2022] DELETE /identity/v3/projects/297ff4b2a2ec44f0b017304e3f08822d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:44.722002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:44.723985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:44.797521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfba70-3dbd-42cc-abec-abd0491b4cd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:44.804133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfba70-3dbd-42cc-abec-abd0491b4cd6 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:44.804339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed1be7-a6c1-4011-95b3-5a8d79c0df0a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:44.809158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e1c77-29e1-4a46-abe2-2b0f80d72e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:44.809812 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e1c77-29e1-4a46-abe2-2b0f80d72e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:44.810091 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e1c77-29e1-4a46-abe2-2b0f80d72e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:44.811843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e1c77-29e1-4a46-abe2-2b0f80d72e33 admin admin] PATH_INFO: `/v3/users/c3606cf6948f419eb31e22ab88a34e70` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:44.813082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e1c77-29e1-4a46-abe2-2b0f80d72e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3606cf6948f419eb31e22ab88a34e70)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:44.815162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e1c77-29e1-4a46-abe2-2b0f80d72e33 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:44.822736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 904/1822] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:44.823367 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:45.014443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-434e1c77-29e1-4a46-abe2-2b0f80d72e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3606cf6948f419eb31e22ab88a34e70'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:45.033408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-434e1c77-29e1-4a46-abe2-2b0f80d72e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3606cf6948f419eb31e22ab88a34e70'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:45.040216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-434e1c77-29e1-4a46-abe2-2b0f80d72e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3606cf6948f419eb31e22ab88a34e70'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:45.112013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-434e1c77-29e1-4a46-abe2-2b0f80d72e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3606cf6948f419eb31e22ab88a34e70'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:45.114403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-434e1c77-29e1-4a46-abe2-2b0f80d72e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3606cf6948f419eb31e22ab88a34e70'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:45.124114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 919/1823] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:44 2022] DELETE /identity/v3/users/c3606cf6948f419eb31e22ab88a34e70 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:45.787617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfba70-3dbd-42cc-abec-abd0491b4cd6 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:45.807684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfba70-3dbd-42cc-abec-abd0491b4cd6 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:45.937804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:45.938740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:45.939198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:45.939607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] PATH_INFO: `/v3/projects/565e20ccd0d145b6be944f10d58a9678` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:45.940880 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=565e20ccd0d145b6be944f10d58a9678)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:45.944334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:46.076407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '565e20ccd0d145b6be944f10d58a9678', 'request_id': 'req-09b7ff18-011b-4106-9a53-796b63316d4a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:46.078440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:46.150861 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '565e20ccd0d145b6be944f10d58a9678', 'request_id': 'req-09b7ff18-011b-4106-9a53-796b63316d4a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:46.161064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '565e20ccd0d145b6be944f10d58a9678', 'request_id': 'req-09b7ff18-011b-4106-9a53-796b63316d4a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:46.187744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] The token cache is being invalidate because project 565e20ccd0d145b6be944f10d58a9678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:46.188212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:46.188814 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-09b7ff18-011b-4106-9a53-796b63316d4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:46.194305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 905/1824] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:45 2022] DELETE /identity/v3/projects/565e20ccd0d145b6be944f10d58a9678 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:46.196385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d48ae-db0b-4e6d-b3ba-f225a402c41d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:46.196953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d48ae-db0b-4e6d-b3ba-f225a402c41d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:46.197265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d48ae-db0b-4e6d-b3ba-f225a402c41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:46.221414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-be8d48ae-db0b-4e6d-b3ba-f225a402c41d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:46.225899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-be8d48ae-db0b-4e6d-b3ba-f225a402c41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:46.299753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 920/1825] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:46.313879 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcad951-00eb-4729-960d-1082486831b8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:46.329544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcad951-00eb-4729-960d-1082486831b8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:46.329544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcad951-00eb-4729-960d-1082486831b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:46.371347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7dcad951-00eb-4729-960d-1082486831b8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:46.372625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7dcad951-00eb-4729-960d-1082486831b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:46.386323 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 906/1826] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:46.391721 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d48ae-db0b-4e6d-b3ba-f225a402c41d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:46.404196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d48ae-db0b-4e6d-b3ba-f225a402c41d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:46.408747 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934fea-c618-4343-bc51-bdcb37af9a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:46.409322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934fea-c618-4343-bc51-bdcb37af9a30 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:46.409650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934fea-c618-4343-bc51-bdcb37af9a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:46.410043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934fea-c618-4343-bc51-bdcb37af9a30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:46.410740 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18934fea-c618-4343-bc51-bdcb37af9a30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:51:46.411057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18934fea-c618-4343-bc51-bdcb37af9a30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:46.412836 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18934fea-c618-4343-bc51-bdcb37af9a30 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:46.430299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 921/1827] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:51:46.431492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcad951-00eb-4729-960d-1082486831b8 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:46.453793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcad951-00eb-4729-960d-1082486831b8 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:46.458937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc7707-e2f3-44f7-b5fb-7c0c29c9f2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:46.459584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc7707-e2f3-44f7-b5fb-7c0c29c9f2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:46.459885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc7707-e2f3-44f7-b5fb-7c0c29c9f2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:46.460502 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc7707-e2f3-44f7-b5fb-7c0c29c9f2b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:46.460784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc7707-e2f3-44f7-b5fb-7c0c29c9f2b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:46.465296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc7707-e2f3-44f7-b5fb-7c0c29c9f2b7 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:46.646176 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 907/1828] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:46 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:51:46.650571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934fea-c618-4343-bc51-bdcb37af9a30 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:46.674897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934fea-c618-4343-bc51-bdcb37af9a30 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:46.789682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abca1e-3204-4c53-911a-ee9034ca3560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:46.790409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abca1e-3204-4c53-911a-ee9034ca3560 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:46.790832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abca1e-3204-4c53-911a-ee9034ca3560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:46.791114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abca1e-3204-4c53-911a-ee9034ca3560 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:46.798448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abca1e-3204-4c53-911a-ee9034ca3560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:46.798448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abca1e-3204-4c53-911a-ee9034ca3560 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:46.888536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 922/1829] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:46 2022] POST /identity/v3/users => generated 287 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:46.892893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc7707-e2f3-44f7-b5fb-7c0c29c9f2b7 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:46.904666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc7707-e2f3-44f7-b5fb-7c0c29c9f2b7 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:46.909782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c629f-a9d8-45c8-9474-afd9bf291368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:46.911199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c629f-a9d8-45c8-9474-afd9bf291368 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:46.911555 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c629f-a9d8-45c8-9474-afd9bf291368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:46.911911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c629f-a9d8-45c8-9474-afd9bf291368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:46.912588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c629f-a9d8-45c8-9474-afd9bf291368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:46.919681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c629f-a9d8-45c8-9474-afd9bf291368 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:46.931209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 908/1830] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:46.935503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abca1e-3204-4c53-911a-ee9034ca3560 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:46.941874 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:46.947868 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abca1e-3204-4c53-911a-ee9034ca3560 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:46.953441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e1c30-f508-44ee-8532-124d2390db85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:46.958400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e1c30-f508-44ee-8532-124d2390db85 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:46.958400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e1c30-f508-44ee-8532-124d2390db85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:46.958400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e1c30-f508-44ee-8532-124d2390db85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:46.958400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e1c30-f508-44ee-8532-124d2390db85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:46.958400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e1c30-f508-44ee-8532-124d2390db85 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:46.971828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 923/1831] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:46.979031 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c629f-a9d8-45c8-9474-afd9bf291368 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:46.999104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c629f-a9d8-45c8-9474-afd9bf291368 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:47.011932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7118fd-abc1-4abf-979c-2a90609554f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:47.012709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7118fd-abc1-4abf-979c-2a90609554f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:47.013093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7118fd-abc1-4abf-979c-2a90609554f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:47.013432 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7118fd-abc1-4abf-979c-2a90609554f1 admin admin] PATH_INFO: `/v3/projects/a2c183e476124f27b34babceaa402699/users/07c864ce8c754cc2a74223e91f223527/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:47.054285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7118fd-abc1-4abf-979c-2a90609554f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2c183e476124f27b34babceaa402699, user_id=07c864ce8c754cc2a74223e91f223527, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:47.057511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7118fd-abc1-4abf-979c-2a90609554f1 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:47.157858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 909/1832] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:46 2022] PUT /identity/v3/projects/a2c183e476124f27b34babceaa402699/users/07c864ce8c754cc2a74223e91f223527/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:47.177014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87b191-7ebd-42aa-ada8-d3faf042fd0b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:47.177526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87b191-7ebd-42aa-ada8-d3faf042fd0b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:47.177811 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87b191-7ebd-42aa-ada8-d3faf042fd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:47.227989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-fd87b191-7ebd-42aa-ada8-d3faf042fd0b None None] MFA Rules not processed for user `07c864ce8c754cc2a74223e91f223527`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:47.231730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fd87b191-7ebd-42aa-ada8-d3faf042fd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:47.376754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 924/1833] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:47.388903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7118fd-abc1-4abf-979c-2a90609554f1 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:47.391002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-de7118fd-abc1-4abf-979c-2a90609554f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:47.493530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7118fd-abc1-4abf-979c-2a90609554f1 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:47.498746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f8732-beab-4a20-8b8b-c5c5451fdf0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:47.499376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8732-beab-4a20-8b8b-c5c5451fdf0e service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:47.499653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8732-beab-4a20-8b8b-c5c5451fdf0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:47.499922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8732-beab-4a20-8b8b-c5c5451fdf0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:47.518639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f8732-beab-4a20-8b8b-c5c5451fdf0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:47.520954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f8732-beab-4a20-8b8b-c5c5451fdf0e service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:47.545302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 910/1834] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:47.565899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87b191-7ebd-42aa-ada8-d3faf042fd0b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:47.645794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87b191-7ebd-42aa-ada8-d3faf042fd0b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:47.650944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:47.651605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:47.651879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:47.652156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] PATH_INFO: `/v3/users/07c864ce8c754cc2a74223e91f223527` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:47.653330 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c864ce8c754cc2a74223e91f223527)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:47.655339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:47.841609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c864ce8c754cc2a74223e91f223527'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:47.857176 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c864ce8c754cc2a74223e91f223527'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:47.866370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c864ce8c754cc2a74223e91f223527'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:47.941066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c864ce8c754cc2a74223e91f223527'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:47.941804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c864ce8c754cc2a74223e91f223527'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:47.949544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 925/1835] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:47 2022] DELETE /identity/v3/users/07c864ce8c754cc2a74223e91f223527 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:47.962871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b85d60-cbfc-4479-a9c5-21b55ff1978a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:47.963489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b85d60-cbfc-4479-a9c5-21b55ff1978a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:47.963489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b85d60-cbfc-4479-a9c5-21b55ff1978a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:48.008977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-39b85d60-cbfc-4479-a9c5-21b55ff1978a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:48.012075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-39b85d60-cbfc-4479-a9c5-21b55ff1978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:48.128588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 911/1836] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:48.143685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:48.145400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:48.226151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826b82c-20ab-45cd-87ea-6314e30d66f4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:48.230800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa844715-b115-4d0a-afbc-2a23c01473c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:48.231898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa844715-b115-4d0a-afbc-2a23c01473c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:48.231898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa844715-b115-4d0a-afbc-2a23c01473c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:48.232159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa844715-b115-4d0a-afbc-2a23c01473c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:48.243514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa844715-b115-4d0a-afbc-2a23c01473c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:48.245015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa844715-b115-4d0a-afbc-2a23c01473c5 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:48.255651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 926/1837] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:48.482017 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:48.767919 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b85d60-cbfc-4479-a9c5-21b55ff1978a None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:48.770307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-39b85d60-cbfc-4479-a9c5-21b55ff1978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:48.787920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b85d60-cbfc-4479-a9c5-21b55ff1978a None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:48.791373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:48.792129 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:48.792472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:48.792807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] PATH_INFO: `/v3/projects/a2c183e476124f27b34babceaa402699` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:48.805536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2c183e476124f27b34babceaa402699)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:48.807416 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:48.846750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:48.912204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939e118-0ced-4e3b-b953-fd7d6ba75c2e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:48.921253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939e118-0ced-4e3b-b953-fd7d6ba75c2e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:48.921253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939e118-0ced-4e3b-b953-fd7d6ba75c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:48.933855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8939e118-0ced-4e3b-b953-fd7d6ba75c2e None None] MFA Rules not processed for user `863bcfa6b8694182b9d163133a141907`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:48.934889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8939e118-0ced-4e3b-b953-fd7d6ba75c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:48.942923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 927/1838] 10.222.0.61 () {58 vars in 1015 bytes} [Mon Aug 29 23:51:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:48.977735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2c183e476124f27b34babceaa402699', 'request_id': 'req-4d54bab9-5547-4042-8fd4-27e5de0b34f3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:48.999966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939e118-0ced-4e3b-b953-fd7d6ba75c2e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:49.002913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8939e118-0ced-4e3b-b953-fd7d6ba75c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:49.014836 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939e118-0ced-4e3b-b953-fd7d6ba75c2e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:49.078019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8bca7-b800-4336-9984-96971427427b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:49.078686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8bca7-b800-4336-9984-96971427427b service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:49.078967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8bca7-b800-4336-9984-96971427427b service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:49.079242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8bca7-b800-4336-9984-96971427427b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:49.088302 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8bca7-b800-4336-9984-96971427427b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:49.089813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8bca7-b800-4336-9984-96971427427b service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:49.104679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 928/1839] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:51:49.105545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8bca7-b800-4336-9984-96971427427b service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:49.119422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8bca7-b800-4336-9984-96971427427b service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:49.124209 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1017898-8860-4573-a957-7dcc1dc256f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:49.124500 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1017898-8860-4573-a957-7dcc1dc256f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:49.124780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1017898-8860-4573-a957-7dcc1dc256f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:49.125053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1017898-8860-4573-a957-7dcc1dc256f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:49.134319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2c183e476124f27b34babceaa402699', 'request_id': 'req-4d54bab9-5547-4042-8fd4-27e5de0b34f3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:49.136204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1017898-8860-4573-a957-7dcc1dc256f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:49.138404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1017898-8860-4573-a957-7dcc1dc256f2 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:49.146703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2c183e476124f27b34babceaa402699', 'request_id': 'req-4d54bab9-5547-4042-8fd4-27e5de0b34f3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:49.151227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 929/1840] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:49.152194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1017898-8860-4573-a957-7dcc1dc256f2 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:49.181209 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1017898-8860-4573-a957-7dcc1dc256f2 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:49.185292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] The token cache is being invalidate because project a2c183e476124f27b34babceaa402699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:49.185895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:49.187980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4c69d-7bc8-49c9-b886-6a86efbb52f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:49.188626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:49.189771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4c69d-7bc8-49c9-b886-6a86efbb52f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:49.190281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4c69d-7bc8-49c9-b886-6a86efbb52f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:49.190790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4c69d-7bc8-49c9-b886-6a86efbb52f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:49.193739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 912/1841] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:48 2022] DELETE /identity/v3/projects/a2c183e476124f27b34babceaa402699 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:49.194053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-31a4c69d-7bc8-49c9-b886-6a86efbb52f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:49.195355 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:49.201256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:49.219817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4c69d-7bc8-49c9-b886-6a86efbb52f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:49.221984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4c69d-7bc8-49c9-b886-6a86efbb52f6 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:49.234154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 930/1842] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:49.236707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202cdf6-8923-49b0-9de7-956fb47d1693 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:49.237219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202cdf6-8923-49b0-9de7-956fb47d1693 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:49.249934 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202cdf6-8923-49b0-9de7-956fb47d1693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:49.250012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54bab9-5547-4042-8fd4-27e5de0b34f3 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:49.261140 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7c8bf-000a-4730-99ec-2daaf921517d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:49.262175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7c8bf-000a-4730-99ec-2daaf921517d service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:49.262710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7c8bf-000a-4730-99ec-2daaf921517d service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:49.263295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7c8bf-000a-4730-99ec-2daaf921517d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:49.263624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3202cdf6-8923-49b0-9de7-956fb47d1693 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:49.265785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3202cdf6-8923-49b0-9de7-956fb47d1693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:49.290456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7c8bf-000a-4730-99ec-2daaf921517d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:49.292032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7c8bf-000a-4730-99ec-2daaf921517d service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:49.303180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 913/1843] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:49.399429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 931/1844] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:49.408570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0349651-7a37-4737-8078-a5a5d6274a49 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:49.408857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0349651-7a37-4737-8078-a5a5d6274a49 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:49.409315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0349651-7a37-4737-8078-a5a5d6274a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:49.432010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f0349651-7a37-4737-8078-a5a5d6274a49 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:49.440777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f0349651-7a37-4737-8078-a5a5d6274a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:49.464457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202cdf6-8923-49b0-9de7-956fb47d1693 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:49.466794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 914/1845] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:49.476008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202cdf6-8923-49b0-9de7-956fb47d1693 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:49.481400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731ec6c-2f99-4ff5-9f46-2165cd5c6e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:49.482187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731ec6c-2f99-4ff5-9f46-2165cd5c6e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:49.482587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731ec6c-2f99-4ff5-9f46-2165cd5c6e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:49.482911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731ec6c-2f99-4ff5-9f46-2165cd5c6e76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:49.483898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731ec6c-2f99-4ff5-9f46-2165cd5c6e76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:51:49.484297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731ec6c-2f99-4ff5-9f46-2165cd5c6e76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:49.488646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731ec6c-2f99-4ff5-9f46-2165cd5c6e76 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:49.507166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0349651-7a37-4737-8078-a5a5d6274a49 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:49.510644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 932/1846] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:51:49.519178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0349651-7a37-4737-8078-a5a5d6274a49 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:49.525775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98a026-fe78-46ba-a81c-fcc5a12aa45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:49.526512 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98a026-fe78-46ba-a81c-fcc5a12aa45d admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:49.526909 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98a026-fe78-46ba-a81c-fcc5a12aa45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:49.527252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98a026-fe78-46ba-a81c-fcc5a12aa45d admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:49.527981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98a026-fe78-46ba-a81c-fcc5a12aa45d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:49.529965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98a026-fe78-46ba-a81c-fcc5a12aa45d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:49.641562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 915/1847] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:49 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:49.645970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731ec6c-2f99-4ff5-9f46-2165cd5c6e76 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:49.660498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731ec6c-2f99-4ff5-9f46-2165cd5c6e76 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:49.782336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71d2ff-c17a-4a16-9fda-bfab6d0a757e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:49.783283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71d2ff-c17a-4a16-9fda-bfab6d0a757e admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:49.783660 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71d2ff-c17a-4a16-9fda-bfab6d0a757e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:49.784055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71d2ff-c17a-4a16-9fda-bfab6d0a757e admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:49.785998 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71d2ff-c17a-4a16-9fda-bfab6d0a757e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:49.789483 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71d2ff-c17a-4a16-9fda-bfab6d0a757e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:49.894065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 933/1848] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:49 2022] POST /identity/v3/users => generated 291 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:49.895863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98a026-fe78-46ba-a81c-fcc5a12aa45d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:49.906737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98a026-fe78-46ba-a81c-fcc5a12aa45d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:49.912002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc0992-4101-4be8-9f33-207e77ce811d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:49.912625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc0992-4101-4be8-9f33-207e77ce811d admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:49.913021 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc0992-4101-4be8-9f33-207e77ce811d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:49.914197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc0992-4101-4be8-9f33-207e77ce811d admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:49.914197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc0992-4101-4be8-9f33-207e77ce811d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:49.915788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc0992-4101-4be8-9f33-207e77ce811d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:49.930338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 916/1849] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:49.936651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71d2ff-c17a-4a16-9fda-bfab6d0a757e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:49.961594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71d2ff-c17a-4a16-9fda-bfab6d0a757e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:49.971306 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cf097-dc79-4743-a0d9-3dc833ba8db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:49.974656 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cf097-dc79-4743-a0d9-3dc833ba8db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:49.975208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cf097-dc79-4743-a0d9-3dc833ba8db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:49.975502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cf097-dc79-4743-a0d9-3dc833ba8db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:49.976194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cf097-dc79-4743-a0d9-3dc833ba8db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:49.977897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cf097-dc79-4743-a0d9-3dc833ba8db3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:49.999137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 934/1850] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:50.003466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc0992-4101-4be8-9f33-207e77ce811d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:50.016462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc0992-4101-4be8-9f33-207e77ce811d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:50.024697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f7dc6-6a9c-4028-9be7-601c745eae7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:50.025870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f7dc6-6a9c-4028-9be7-601c745eae7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:50.026953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f7dc6-6a9c-4028-9be7-601c745eae7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:50.027426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f7dc6-6a9c-4028-9be7-601c745eae7e admin admin] PATH_INFO: `/v3/projects/7524ddcc684c4afe8fac71722c6f59ba/users/46e697e14f5a4d6288e358b7365c9498/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:50.054630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f7dc6-6a9c-4028-9be7-601c745eae7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7524ddcc684c4afe8fac71722c6f59ba, user_id=46e697e14f5a4d6288e358b7365c9498, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:50.054630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f7dc6-6a9c-4028-9be7-601c745eae7e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:50.162048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 917/1851] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:50 2022] PUT /identity/v3/projects/7524ddcc684c4afe8fac71722c6f59ba/users/46e697e14f5a4d6288e358b7365c9498/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:50.173149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5b467-e8dd-4a03-a626-e61baee5ad60 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:50.173482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5b467-e8dd-4a03-a626-e61baee5ad60 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:50.173785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5b467-e8dd-4a03-a626-e61baee5ad60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:50.207785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-26c5b467-e8dd-4a03-a626-e61baee5ad60 None None] MFA Rules not processed for user `46e697e14f5a4d6288e358b7365c9498`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:50.209090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-26c5b467-e8dd-4a03-a626-e61baee5ad60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:50.294366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:50.308033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 935/1852] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:50.312727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f7dc6-6a9c-4028-9be7-601c745eae7e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:50.383026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5b467-e8dd-4a03-a626-e61baee5ad60 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:50.384520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-26c5b467-e8dd-4a03-a626-e61baee5ad60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:50.490565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f7dc6-6a9c-4028-9be7-601c745eae7e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:50.499806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a6a8b-2962-4dcd-8709-8f96a697c74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:50.500518 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a6a8b-2962-4dcd-8709-8f96a697c74e admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:50.500888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a6a8b-2962-4dcd-8709-8f96a697c74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:50.501233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a6a8b-2962-4dcd-8709-8f96a697c74e admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:50.502644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a6a8b-2962-4dcd-8709-8f96a697c74e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:50.505511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a6a8b-2962-4dcd-8709-8f96a697c74e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:50.529109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5b467-e8dd-4a03-a626-e61baee5ad60 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:50.542985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f1e78-eccc-4b6e-b6cd-f87be6ebefa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:50.543992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f1e78-eccc-4b6e-b6cd-f87be6ebefa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:50.544458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f1e78-eccc-4b6e-b6cd-f87be6ebefa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:50.544923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f1e78-eccc-4b6e-b6cd-f87be6ebefa4 admin admin] PATH_INFO: `/v3/users/14cdcf58839b4a31b0cc969aacfe9b0f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:50.546608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f1e78-eccc-4b6e-b6cd-f87be6ebefa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14cdcf58839b4a31b0cc969aacfe9b0f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:50.548970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f1e78-eccc-4b6e-b6cd-f87be6ebefa4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:50.640208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 918/1853] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:50 2022] POST /identity/v3/projects => generated 383 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:50.645451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a6a8b-2962-4dcd-8709-8f96a697c74e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:50.705329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:50.772939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-714f1e78-eccc-4b6e-b6cd-f87be6ebefa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14cdcf58839b4a31b0cc969aacfe9b0f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:50.773094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a6a8b-2962-4dcd-8709-8f96a697c74e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:50.782143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a577e-b638-49ee-a320-ab0575ba01b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:50.784129 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a577e-b638-49ee-a320-ab0575ba01b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:50.784718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a577e-b638-49ee-a320-ab0575ba01b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:50.785332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a577e-b638-49ee-a320-ab0575ba01b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:50.786911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a577e-b638-49ee-a320-ab0575ba01b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:50.790105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a577e-b638-49ee-a320-ab0575ba01b2 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:50.797811 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-714f1e78-eccc-4b6e-b6cd-f87be6ebefa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14cdcf58839b4a31b0cc969aacfe9b0f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:50.821607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-714f1e78-eccc-4b6e-b6cd-f87be6ebefa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14cdcf58839b4a31b0cc969aacfe9b0f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:50.905061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:50.906984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-714f1e78-eccc-4b6e-b6cd-f87be6ebefa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14cdcf58839b4a31b0cc969aacfe9b0f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:50.907639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-714f1e78-eccc-4b6e-b6cd-f87be6ebefa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14cdcf58839b4a31b0cc969aacfe9b0f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:50.926901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 936/1854] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:50 2022] DELETE /identity/v3/users/14cdcf58839b4a31b0cc969aacfe9b0f => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:50.929056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5aec66-05d8-410c-a57f-c4685d70a04f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:50.929345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5aec66-05d8-410c-a57f-c4685d70a04f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:50.929616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5aec66-05d8-410c-a57f-c4685d70a04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:50.930911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 919/1855] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:50 2022] POST /identity/v3/users => generated 292 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:50.934469 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a577e-b638-49ee-a320-ab0575ba01b2 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:50.940652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-fd0a577e-b638-49ee-a320-ab0575ba01b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:50.979218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2b5aec66-05d8-410c-a57f-c4685d70a04f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:50.980499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2b5aec66-05d8-410c-a57f-c4685d70a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:50.997491 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a577e-b638-49ee-a320-ab0575ba01b2 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:51.074144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 937/1856] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:51.133617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469b92d-acc0-4330-9316-cfa889f0f0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:51.134820 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469b92d-acc0-4330-9316-cfa889f0f0cb admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:51.135154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469b92d-acc0-4330-9316-cfa889f0f0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:51.135562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469b92d-acc0-4330-9316-cfa889f0f0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:51.136233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469b92d-acc0-4330-9316-cfa889f0f0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:51.138206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469b92d-acc0-4330-9316-cfa889f0f0cb admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:51.154242 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 920/1857] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:51.155954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5aec66-05d8-410c-a57f-c4685d70a04f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:51.171391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5aec66-05d8-410c-a57f-c4685d70a04f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:51.178736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ee0e5-ccc2-4d2f-83c1-135bcd845008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:51.178736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ee0e5-ccc2-4d2f-83c1-135bcd845008 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:51.179181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ee0e5-ccc2-4d2f-83c1-135bcd845008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:51.180103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ee0e5-ccc2-4d2f-83c1-135bcd845008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:51.180860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ee0e5-ccc2-4d2f-83c1-135bcd845008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:51.182795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ee0e5-ccc2-4d2f-83c1-135bcd845008 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:51.209685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 938/1858] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:51.213962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469b92d-acc0-4330-9316-cfa889f0f0cb admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:51.227942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469b92d-acc0-4330-9316-cfa889f0f0cb admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:51.236781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42b61b-c169-47e9-88ad-1a72db66f5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:51.237650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42b61b-c169-47e9-88ad-1a72db66f5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:51.237746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42b61b-c169-47e9-88ad-1a72db66f5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:51.238215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42b61b-c169-47e9-88ad-1a72db66f5dc admin admin] PATH_INFO: `/v3/projects/65d89c9e4d1a417f86fd541ca254ea9f/users/79f3c10ccf8b4f4bb10f506ef6af3ada/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:51.280735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42b61b-c169-47e9-88ad-1a72db66f5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d89c9e4d1a417f86fd541ca254ea9f, user_id=79f3c10ccf8b4f4bb10f506ef6af3ada, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:51.284471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42b61b-c169-47e9-88ad-1a72db66f5dc admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:51.298397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40525d7d-018f-44f8-b966-bb3e7330447b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:51.299501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40525d7d-018f-44f8-b966-bb3e7330447b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:51.300401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40525d7d-018f-44f8-b966-bb3e7330447b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:51.304538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 939/1859] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:51:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:51:51.314977 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40525d7d-018f-44f8-b966-bb3e7330447b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:51.317078 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-40525d7d-018f-44f8-b966-bb3e7330447b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:51.354100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 921/1860] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:51 2022] PUT /identity/v3/projects/65d89c9e4d1a417f86fd541ca254ea9f/users/79f3c10ccf8b4f4bb10f506ef6af3ada/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:51.366547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d2bce-7ad6-4241-9e1a-8d73349788af None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:51.367176 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d2bce-7ad6-4241-9e1a-8d73349788af None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:51.367760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d2bce-7ad6-4241-9e1a-8d73349788af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:51.450003 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-290d2bce-7ad6-4241-9e1a-8d73349788af None None] MFA Rules not processed for user `79f3c10ccf8b4f4bb10f506ef6af3ada`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:51.451742 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-290d2bce-7ad6-4241-9e1a-8d73349788af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:51.460443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40525d7d-018f-44f8-b966-bb3e7330447b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:51.586557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30623c52-99cc-4ab8-9e34-36e7a64e3d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:51.587191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30623c52-99cc-4ab8-9e34-36e7a64e3d16 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:51.587462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30623c52-99cc-4ab8-9e34-36e7a64e3d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:51.587895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30623c52-99cc-4ab8-9e34-36e7a64e3d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:51.632515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 922/1861] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:51:51.635877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632dde3-3d98-49ed-8744-c8dd27a5a833 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:51.638598 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632dde3-3d98-49ed-8744-c8dd27a5a833 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:51.639057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632dde3-3d98-49ed-8744-c8dd27a5a833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:51.678584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-a632dde3-3d98-49ed-8744-c8dd27a5a833 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:51.679837 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a632dde3-3d98-49ed-8744-c8dd27a5a833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:51.712931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30623c52-99cc-4ab8-9e34-36e7a64e3d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:51.715178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30623c52-99cc-4ab8-9e34-36e7a64e3d16 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:51.732661 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:51.749319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 940/1862] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:51.750376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30623c52-99cc-4ab8-9e34-36e7a64e3d16 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:51.764095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30623c52-99cc-4ab8-9e34-36e7a64e3d16 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:51.777106 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9e09c-6d46-441f-83b6-4224bce1eb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:51.777936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9e09c-6d46-441f-83b6-4224bce1eb6b admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:51.778276 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9e09c-6d46-441f-83b6-4224bce1eb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:51.778647 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9e09c-6d46-441f-83b6-4224bce1eb6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:51.779581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9e09c-6d46-441f-83b6-4224bce1eb6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:51.783382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9e09c-6d46-441f-83b6-4224bce1eb6b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:51.831124 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 923/1863] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:51:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:51.894324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4d5e0-12d1-4385-8fbd-4e366efc8828 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:51.894801 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4d5e0-12d1-4385-8fbd-4e366efc8828 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:51.895145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4d5e0-12d1-4385-8fbd-4e366efc8828 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:51.897651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 924/1864] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:51:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:51:51.939558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee9f7e-aff1-4c9a-b6c7-4e7140f857aa None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:51.955011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee9f7e-aff1-4c9a-b6c7-4e7140f857aa None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:51.958113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 941/1865] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:51 2022] POST /identity/v3/projects => generated 385 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:51.959131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee9f7e-aff1-4c9a-b6c7-4e7140f857aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:51.962481 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9e09c-6d46-441f-83b6-4224bce1eb6b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:52.000873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-41ee9f7e-aff1-4c9a-b6c7-4e7140f857aa None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:52.002308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-41ee9f7e-aff1-4c9a-b6c7-4e7140f857aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:52.055301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:52.092466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 925/1866] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:51:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:52.147300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9e09c-6d46-441f-83b6-4224bce1eb6b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:52.159328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0435946-2f4c-4048-87dc-ad4b10462b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:52.159328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0435946-2f4c-4048-87dc-ad4b10462b37 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:52.159520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0435946-2f4c-4048-87dc-ad4b10462b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:52.159652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0435946-2f4c-4048-87dc-ad4b10462b37 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:52.160392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0435946-2f4c-4048-87dc-ad4b10462b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:52.162399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0435946-2f4c-4048-87dc-ad4b10462b37 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:52.311594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 942/1867] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:51 2022] POST /identity/v3/users => generated 299 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:52.315452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee9f7e-aff1-4c9a-b6c7-4e7140f857aa None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:52.333231 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee9f7e-aff1-4c9a-b6c7-4e7140f857aa None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:52.343159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598df98-6a07-4590-9347-e0b971ae6b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:52.343159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598df98-6a07-4590-9347-e0b971ae6b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:52.343159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598df98-6a07-4590-9347-e0b971ae6b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:52.343159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598df98-6a07-4590-9347-e0b971ae6b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:52.343521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598df98-6a07-4590-9347-e0b971ae6b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:52.346467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598df98-6a07-4590-9347-e0b971ae6b4e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:52.360035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 926/1868] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:51:52.364702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0435946-2f4c-4048-87dc-ad4b10462b37 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:52.376431 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0435946-2f4c-4048-87dc-ad4b10462b37 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:52.382684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286c664-cd6a-4190-b110-f8914662d4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:52.383842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286c664-cd6a-4190-b110-f8914662d4b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:52.384316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286c664-cd6a-4190-b110-f8914662d4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:52.384996 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286c664-cd6a-4190-b110-f8914662d4b6 admin admin] PATH_INFO: `/v3/projects/0c64bb8c630c4067adfe31c69d59b8be/users/de761a9e0a6c4b74a73960c431c2c4b4/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:52.404367 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286c664-cd6a-4190-b110-f8914662d4b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c64bb8c630c4067adfe31c69d59b8be, user_id=de761a9e0a6c4b74a73960c431c2c4b4, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:52.407749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286c664-cd6a-4190-b110-f8914662d4b6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:52.464663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598df98-6a07-4590-9347-e0b971ae6b4e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:52.468907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3598df98-6a07-4590-9347-e0b971ae6b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:52.497845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598df98-6a07-4590-9347-e0b971ae6b4e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:52.498467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 943/1869] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:52 2022] PUT /identity/v3/projects/0c64bb8c630c4067adfe31c69d59b8be/users/de761a9e0a6c4b74a73960c431c2c4b4/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:52.500101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286c664-cd6a-4190-b110-f8914662d4b6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:52.601770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286c664-cd6a-4190-b110-f8914662d4b6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:52.607047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbade8-0964-4177-b378-2c7bc0281743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:52.607717 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbade8-0964-4177-b378-2c7bc0281743 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:52.608353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbade8-0964-4177-b378-2c7bc0281743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:52.608353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbade8-0964-4177-b378-2c7bc0281743 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:52.608939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbade8-0964-4177-b378-2c7bc0281743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:52.610942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbade8-0964-4177-b378-2c7bc0281743 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:52.627936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 944/1870] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:52.629519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbade8-0964-4177-b378-2c7bc0281743 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:52.632011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d60880-13c1-40a3-b7e7-1085a28620fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:52.632927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d60880-13c1-40a3-b7e7-1085a28620fb service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:52.633409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d60880-13c1-40a3-b7e7-1085a28620fb service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:52.633880 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d60880-13c1-40a3-b7e7-1085a28620fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:52.642951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbade8-0964-4177-b378-2c7bc0281743 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:52.649430 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d60880-13c1-40a3-b7e7-1085a28620fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:52.651822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21adf818-34cc-4573-aff8-6fdcec0a496f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:52.651822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adf818-34cc-4573-aff8-6fdcec0a496f admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:52.652033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adf818-34cc-4573-aff8-6fdcec0a496f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:52.652339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adf818-34cc-4573-aff8-6fdcec0a496f admin admin] PATH_INFO: `/v3/projects/0c64bb8c630c4067adfe31c69d59b8be/users/de761a9e0a6c4b74a73960c431c2c4b4/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:52.652657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d60880-13c1-40a3-b7e7-1085a28620fb service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:52.655103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21adf818-34cc-4573-aff8-6fdcec0a496f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c64bb8c630c4067adfe31c69d59b8be, user_id=de761a9e0a6c4b74a73960c431c2c4b4, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:52.661261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21adf818-34cc-4573-aff8-6fdcec0a496f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:52.668369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 927/1871] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:52.699461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:52.703473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 945/1872] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:52 2022] PUT /identity/v3/projects/0c64bb8c630c4067adfe31c69d59b8be/users/de761a9e0a6c4b74a73960c431c2c4b4/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:52.704716 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d60880-13c1-40a3-b7e7-1085a28620fb service nova] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:52.707058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-24d60880-13c1-40a3-b7e7-1085a28620fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:52.713480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5705ce-0a2e-4ea0-b1bd-d075cb92b8aa None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:52.714040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5705ce-0a2e-4ea0-b1bd-d075cb92b8aa None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:52.714550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5705ce-0a2e-4ea0-b1bd-d075cb92b8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:52.734609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d60880-13c1-40a3-b7e7-1085a28620fb service nova] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:52.796925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9d5705ce-0a2e-4ea0-b1bd-d075cb92b8aa None None] MFA Rules not processed for user `de761a9e0a6c4b74a73960c431c2c4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:52.799436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9d5705ce-0a2e-4ea0-b1bd-d075cb92b8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:52.890104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8661379-e44e-4bf6-94a4-bd9b011ff119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:52.890753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8661379-e44e-4bf6-94a4-bd9b011ff119 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:52.891031 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8661379-e44e-4bf6-94a4-bd9b011ff119 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:52.891374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8661379-e44e-4bf6-94a4-bd9b011ff119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:52.899597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:52.925318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8661379-e44e-4bf6-94a4-bd9b011ff119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:52.928581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8661379-e44e-4bf6-94a4-bd9b011ff119 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:52.946054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 928/1873] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:51:52.969362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 946/1874] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:51:52.983075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8661379-e44e-4bf6-94a4-bd9b011ff119 service nova] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:52.998670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e8661379-e44e-4bf6-94a4-bd9b011ff119 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:53.081070 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8661379-e44e-4bf6-94a4-bd9b011ff119 service nova] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:53.085363 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf56328-3af9-4772-a978-1a488f66fd40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:53.085994 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56328-3af9-4772-a978-1a488f66fd40 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:53.086306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56328-3af9-4772-a978-1a488f66fd40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:53.090594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56328-3af9-4772-a978-1a488f66fd40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:53.090594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9bf56328-3af9-4772-a978-1a488f66fd40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:53.168252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf56328-3af9-4772-a978-1a488f66fd40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:53.169880 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf56328-3af9-4772-a978-1a488f66fd40 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:53.182059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 929/1875] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:53.390134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5705ce-0a2e-4ea0-b1bd-d075cb92b8aa None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:53.391932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9d5705ce-0a2e-4ea0-b1bd-d075cb92b8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:53.480391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5705ce-0a2e-4ea0-b1bd-d075cb92b8aa None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:53.486307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:53.487068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:53.488083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:53.488083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] PATH_INFO: `/v3/projects/f95577114e5e44dc990567c47ed5f838` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:53.489566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f95577114e5e44dc990567c47ed5f838)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:53.498660 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:53.661887 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f95577114e5e44dc990567c47ed5f838', 'request_id': 'req-e6f2696c-3114-46a7-8bae-34db69566f7c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:53.667376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f95577114e5e44dc990567c47ed5f838', 'request_id': 'req-e6f2696c-3114-46a7-8bae-34db69566f7c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:53.722351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f95577114e5e44dc990567c47ed5f838', 'request_id': 'req-e6f2696c-3114-46a7-8bae-34db69566f7c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:53.742374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] The token cache is being invalidate because project f95577114e5e44dc990567c47ed5f838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:53.743741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:53.744781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:53.748633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:53.754606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 947/1876] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:53 2022] DELETE /identity/v3/projects/f95577114e5e44dc990567c47ed5f838 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:53.755193 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf56328-3af9-4772-a978-1a488f66fd40 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:53.766996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf56328-3af9-4772-a978-1a488f66fd40 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:53.811225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:53.844251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200b6f8-ee6d-4606-af0e-d0800969e141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:53.844948 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200b6f8-ee6d-4606-af0e-d0800969e141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:53.845232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200b6f8-ee6d-4606-af0e-d0800969e141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:53.845511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200b6f8-ee6d-4606-af0e-d0800969e141 admin admin] PATH_INFO: `/v3/users/6e9d3586f2b54423bb90091712975208` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:53.847028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200b6f8-ee6d-4606-af0e-d0800969e141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9d3586f2b54423bb90091712975208)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:53.850568 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200b6f8-ee6d-4606-af0e-d0800969e141 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:53.952781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:53.954300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:53.969531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2696c-3114-46a7-8bae-34db69566f7c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:53.975896 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:53.979278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:53.979278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:53.979278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] PATH_INFO: `/v3/users/4f4620effdc8492da36fedeb3c5812bd` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:53.979278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f4620effdc8492da36fedeb3c5812bd)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:53.980281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:54.063783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-0200b6f8-ee6d-4606-af0e-d0800969e141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9d3586f2b54423bb90091712975208'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:54.079097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-0200b6f8-ee6d-4606-af0e-d0800969e141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9d3586f2b54423bb90091712975208'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:54.087273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-0200b6f8-ee6d-4606-af0e-d0800969e141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9d3586f2b54423bb90091712975208'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:54.187993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f4620effdc8492da36fedeb3c5812bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:54.204046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-0200b6f8-ee6d-4606-af0e-d0800969e141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9d3586f2b54423bb90091712975208'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:54.204642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-0200b6f8-ee6d-4606-af0e-d0800969e141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9d3586f2b54423bb90091712975208'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:54.223923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 930/1877] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:53 2022] DELETE /identity/v3/users/6e9d3586f2b54423bb90091712975208 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:54.224253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f4620effdc8492da36fedeb3c5812bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:54.241465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f4620effdc8492da36fedeb3c5812bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:54.340005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f4620effdc8492da36fedeb3c5812bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:54.340688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f4620effdc8492da36fedeb3c5812bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:54.355324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 948/1878] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:53 2022] DELETE /identity/v3/users/4f4620effdc8492da36fedeb3c5812bd => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:54.361735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99f5d-84ae-4d49-aa6f-b5407296764f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:54.362150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99f5d-84ae-4d49-aa6f-b5407296764f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:54.362536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99f5d-84ae-4d49-aa6f-b5407296764f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:54.390546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-21a99f5d-84ae-4d49-aa6f-b5407296764f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:54.391971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-21a99f5d-84ae-4d49-aa6f-b5407296764f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:54.487712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 931/1879] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:54.557315 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:54.559929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:54.632356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64c95f-7dce-4c42-87d2-dec4b8f56f55 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:54.636373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99642fd2-7801-4d1c-b758-0722342cd1a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:54.637314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99642fd2-7801-4d1c-b758-0722342cd1a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:54.637771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99642fd2-7801-4d1c-b758-0722342cd1a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:54.638044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99642fd2-7801-4d1c-b758-0722342cd1a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:54.648825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99642fd2-7801-4d1c-b758-0722342cd1a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:54.650285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99642fd2-7801-4d1c-b758-0722342cd1a5 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:54.661924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 949/1880] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:55.452592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a99f5d-84ae-4d49-aa6f-b5407296764f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:55.454694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-21a99f5d-84ae-4d49-aa6f-b5407296764f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:55.563420 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a99f5d-84ae-4d49-aa6f-b5407296764f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:55.569453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d1805-29ea-4e7c-9144-bc7b59024961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:55.570197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d1805-29ea-4e7c-9144-bc7b59024961 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:55.570691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d1805-29ea-4e7c-9144-bc7b59024961 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:55.571720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d1805-29ea-4e7c-9144-bc7b59024961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:55.574011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9d8d1805-29ea-4e7c-9144-bc7b59024961 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:55.587360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:55.669813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d1805-29ea-4e7c-9144-bc7b59024961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:55.684156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d1805-29ea-4e7c-9144-bc7b59024961 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:55.704818 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 932/1881] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:56.018066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99642fd2-7801-4d1c-b758-0722342cd1a5 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:56.020470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-99642fd2-7801-4d1c-b758-0722342cd1a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:56.039049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99642fd2-7801-4d1c-b758-0722342cd1a5 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:56.046232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:56.047213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:56.047627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:56.048222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] PATH_INFO: `/v3/projects/1c9ea9072e574b2f9604795ed49f3de7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:56.049393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c9ea9072e574b2f9604795ed49f3de7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:56.052546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:56.114105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d1805-29ea-4e7c-9144-bc7b59024961 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:56.118745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9d8d1805-29ea-4e7c-9144-bc7b59024961 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:56.165401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:56.188009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d1805-29ea-4e7c-9144-bc7b59024961 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:56.201022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c9ea9072e574b2f9604795ed49f3de7', 'request_id': 'req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:56.212793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c9ea9072e574b2f9604795ed49f3de7', 'request_id': 'req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:56.310884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:56.312198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:56.312498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:56.312772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] PATH_INFO: `/v3/projects/e7c9a7f5b50547d682e8db3e114a7b9c` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:56.315064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c9a7f5b50547d682e8db3e114a7b9c)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:56.319621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:56.636512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c9ea9072e574b2f9604795ed49f3de7', 'request_id': 'req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:56.649983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] The token cache is being invalidate because project 1c9ea9072e574b2f9604795ed49f3de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:56.650283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:56.650967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:56.652493 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 950/1882] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:56 2022] DELETE /identity/v3/projects/1c9ea9072e574b2f9604795ed49f3de7 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:56.656624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:56.658066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:56.672014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058a324-5ea0-45dc-96d9-dd01e75a6eb0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:56.677915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3629cc-7a57-4635-baa3-936a0f9624b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:56.678637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3629cc-7a57-4635-baa3-936a0f9624b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:56.678929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3629cc-7a57-4635-baa3-936a0f9624b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:56.679228 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3629cc-7a57-4635-baa3-936a0f9624b7 admin admin] PATH_INFO: `/v3/users/e18acfc590734fb8a76bb5584d443b4c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:56.680473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3629cc-7a57-4635-baa3-936a0f9624b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18acfc590734fb8a76bb5584d443b4c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:56.682945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3629cc-7a57-4635-baa3-936a0f9624b7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:56.872394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c9a7f5b50547d682e8db3e114a7b9c', 'request_id': 'req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:56.946544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c9a7f5b50547d682e8db3e114a7b9c', 'request_id': 'req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:56.955107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c9a7f5b50547d682e8db3e114a7b9c', 'request_id': 'req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:56.980041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] The token cache is being invalidate because project e7c9a7f5b50547d682e8db3e114a7b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:56.980338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:56.981587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:56.983449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 933/1883] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:56 2022] DELETE /identity/v3/projects/e7c9a7f5b50547d682e8db3e114a7b9c => generated 0 bytes in 870 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:56.989832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:56.991723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:57.036134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2b3629cc-7a57-4635-baa3-936a0f9624b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18acfc590734fb8a76bb5584d443b4c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:57.046380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:57.050461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2b3629cc-7a57-4635-baa3-936a0f9624b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18acfc590734fb8a76bb5584d443b4c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:57.059944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2b3629cc-7a57-4635-baa3-936a0f9624b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18acfc590734fb8a76bb5584d443b4c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:57.071012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2f3f2-1de3-4722-9ebe-9217d9a26816 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:57.076858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862812f-2140-499f-a2e8-98d8fc57f74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:57.077540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862812f-2140-499f-a2e8-98d8fc57f74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:57.077804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862812f-2140-499f-a2e8-98d8fc57f74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:57.093645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862812f-2140-499f-a2e8-98d8fc57f74f admin admin] PATH_INFO: `/v3/users/7e3f33b904f9482a91b181d4d967efad` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:57.093645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b862812f-2140-499f-a2e8-98d8fc57f74f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3f33b904f9482a91b181d4d967efad)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:57.093809 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b862812f-2140-499f-a2e8-98d8fc57f74f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:57.145762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2b3629cc-7a57-4635-baa3-936a0f9624b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18acfc590734fb8a76bb5584d443b4c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:57.147509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2b3629cc-7a57-4635-baa3-936a0f9624b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18acfc590734fb8a76bb5584d443b4c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:57.160642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 951/1884] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:56 2022] DELETE /identity/v3/users/e18acfc590734fb8a76bb5584d443b4c => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:57.296813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b862812f-2140-499f-a2e8-98d8fc57f74f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3f33b904f9482a91b181d4d967efad'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:57.317035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b862812f-2140-499f-a2e8-98d8fc57f74f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3f33b904f9482a91b181d4d967efad'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:57.331877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b862812f-2140-499f-a2e8-98d8fc57f74f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3f33b904f9482a91b181d4d967efad'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:57.401697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b862812f-2140-499f-a2e8-98d8fc57f74f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3f33b904f9482a91b181d4d967efad'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:57.404087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b862812f-2140-499f-a2e8-98d8fc57f74f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3f33b904f9482a91b181d4d967efad'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:57.411257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 934/1885] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:51:56 2022] DELETE /identity/v3/users/7e3f33b904f9482a91b181d4d967efad => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:57.726940 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a2c3b-24b5-4b95-807f-08171d001708 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:57.727274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a2c3b-24b5-4b95-807f-08171d001708 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:57.727500 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a2c3b-24b5-4b95-807f-08171d001708 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:57.729961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 952/1886] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:51:57.755496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725db14d-047c-43a7-a9c5-96064dfedfd6 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:57.756114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725db14d-047c-43a7-a9c5-96064dfedfd6 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:57.756653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725db14d-047c-43a7-a9c5-96064dfedfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:57.804941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-725db14d-047c-43a7-a9c5-96064dfedfd6 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:57.807375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-725db14d-047c-43a7-a9c5-96064dfedfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:57.831472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a2c3b-24b5-4b95-807f-08171d001708 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:57.834738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-981a2c3b-24b5-4b95-807f-08171d001708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:58.006167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 935/1887] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:51:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:58.015397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a2c3b-24b5-4b95-807f-08171d001708 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:58.025767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:58.026779 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.027348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.027824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] PATH_INFO: `/v3/projects/59bc3058364f4d418cd7757164f5e254` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.030220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=59bc3058364f4d418cd7757164f5e254)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:58.039202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:58.101441 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725db14d-047c-43a7-a9c5-96064dfedfd6 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:58.103480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-725db14d-047c-43a7-a9c5-96064dfedfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:58.194385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:58.201471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725db14d-047c-43a7-a9c5-96064dfedfd6 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:58.215689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59bc3058364f4d418cd7757164f5e254', 'request_id': 'req-04943a80-75bc-4744-bdd5-545f1e2de98e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:58.217862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59bc3058364f4d418cd7757164f5e254', 'request_id': 'req-04943a80-75bc-4744-bdd5-545f1e2de98e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:58.259491 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:58.264357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80bde2-79c5-4ada-bfa2-813e31de3622 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:58.265010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80bde2-79c5-4ada-bfa2-813e31de3622 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.266965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80bde2-79c5-4ada-bfa2-813e31de3622 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.266965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80bde2-79c5-4ada-bfa2-813e31de3622 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.267421 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3b80bde2-79c5-4ada-bfa2-813e31de3622 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:58.288136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59bc3058364f4d418cd7757164f5e254', 'request_id': 'req-04943a80-75bc-4744-bdd5-545f1e2de98e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:58.305776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] The token cache is being invalidate because project 59bc3058364f4d418cd7757164f5e254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:58.306062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:58.307031 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:58.308978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 953/1888] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:57 2022] DELETE /identity/v3/projects/59bc3058364f4d418cd7757164f5e254 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:58.314274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:58.317733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:58.345745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80bde2-79c5-4ada-bfa2-813e31de3622 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:58.351999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80bde2-79c5-4ada-bfa2-813e31de3622 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:58.362596 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 936/1889] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:51:58.364430 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80bde2-79c5-4ada-bfa2-813e31de3622 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:58.366120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3b80bde2-79c5-4ada-bfa2-813e31de3622 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:58.434015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04943a80-75bc-4744-bdd5-545f1e2de98e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:58.448758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b59a80-3085-4ded-b4d8-e1397b68fee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:58.449385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b59a80-3085-4ded-b4d8-e1397b68fee5 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.449660 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b59a80-3085-4ded-b4d8-e1397b68fee5 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.449930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b59a80-3085-4ded-b4d8-e1397b68fee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.451509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80bde2-79c5-4ada-bfa2-813e31de3622 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:58.451723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-37b59a80-3085-4ded-b4d8-e1397b68fee5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:58.456378 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:58.457085 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.457435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.457714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] PATH_INFO: `/v3/projects/e23dd36834a2481187621884d73a21b0` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.459058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e23dd36834a2481187621884d73a21b0)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:58.461041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:58.464533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b59a80-3085-4ded-b4d8-e1397b68fee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:58.466157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b59a80-3085-4ded-b4d8-e1397b68fee5 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:58.477788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 954/1890] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:51:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:51:58.487994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54967c9-f1b5-4388-a51d-85aa7993216f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.489825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54967c9-f1b5-4388-a51d-85aa7993216f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.489825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54967c9-f1b5-4388-a51d-85aa7993216f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.517295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e54967c9-f1b5-4388-a51d-85aa7993216f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:58.517295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e54967c9-f1b5-4388-a51d-85aa7993216f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:58.532814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 955/1891] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:58.548529 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117311bf-4506-45f4-8149-a68f3c1894e9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.549012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117311bf-4506-45f4-8149-a68f3c1894e9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.549212 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117311bf-4506-45f4-8149-a68f3c1894e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.560167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e23dd36834a2481187621884d73a21b0', 'request_id': 'req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:58.571809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-117311bf-4506-45f4-8149-a68f3c1894e9 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:58.573082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-117311bf-4506-45f4-8149-a68f3c1894e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:58.653947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e23dd36834a2481187621884d73a21b0', 'request_id': 'req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:58.663629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e23dd36834a2481187621884d73a21b0', 'request_id': 'req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:58.668491 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 956/1892] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:51:58.672977 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117311bf-4506-45f4-8149-a68f3c1894e9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:58.683916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] The token cache is being invalidate because project e23dd36834a2481187621884d73a21b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:51:58.683916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:58.685577 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bf2945ae-fd86-4a2f-baf0-6300fc4ff65d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:51:58.688528 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117311bf-4506-45f4-8149-a68f3c1894e9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:58.693376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 937/1893] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:51:58 2022] DELETE /identity/v3/projects/e23dd36834a2481187621884d73a21b0 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:58.695055 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcdc9d-9e8e-432b-b604-7930b2c57431 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.695411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcdc9d-9e8e-432b-b604-7930b2c57431 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.695756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcdc9d-9e8e-432b-b604-7930b2c57431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.698623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c332858-e46d-418e-ad0a-c34f3045f700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:58.699546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c332858-e46d-418e-ad0a-c34f3045f700 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.699936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c332858-e46d-418e-ad0a-c34f3045f700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.700261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c332858-e46d-418e-ad0a-c34f3045f700 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.701145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c332858-e46d-418e-ad0a-c34f3045f700 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:51:58.701453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c332858-e46d-418e-ad0a-c34f3045f700 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:58.703898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c332858-e46d-418e-ad0a-c34f3045f700 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:58.719553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-16fcdc9d-9e8e-432b-b604-7930b2c57431 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:58.720969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-16fcdc9d-9e8e-432b-b604-7930b2c57431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:58.733906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 957/1894] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:51:58.740595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c332858-e46d-418e-ad0a-c34f3045f700 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:58.745720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3c332858-e46d-418e-ad0a-c34f3045f700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:58.755823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 938/1895] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:58.758810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43dd6-904a-4e8c-9e1a-591e0adc5b45 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.759210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43dd6-904a-4e8c-9e1a-591e0adc5b45 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.759584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43dd6-904a-4e8c-9e1a-591e0adc5b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.764985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c332858-e46d-418e-ad0a-c34f3045f700 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:58.771044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b145d-5fdb-4654-aad8-91134429fb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:58.771723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b145d-5fdb-4654-aad8-91134429fb31 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.772094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b145d-5fdb-4654-aad8-91134429fb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.772367 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b145d-5fdb-4654-aad8-91134429fb31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.773189 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b145d-5fdb-4654-aad8-91134429fb31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:58.775365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b145d-5fdb-4654-aad8-91134429fb31 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:58.789923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-b8d43dd6-904a-4e8c-9e1a-591e0adc5b45 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:58.791478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b8d43dd6-904a-4e8c-9e1a-591e0adc5b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:58.802412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 939/1896] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:58.807067 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d43dd6-904a-4e8c-9e1a-591e0adc5b45 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:58.818953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d43dd6-904a-4e8c-9e1a-591e0adc5b45 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:58.824214 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f44d24-332c-4e29-ad08-579919b65509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:58.825813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f44d24-332c-4e29-ad08-579919b65509 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.825813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f44d24-332c-4e29-ad08-579919b65509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.825813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f44d24-332c-4e29-ad08-579919b65509 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.826405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f44d24-332c-4e29-ad08-579919b65509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:51:58.826802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f44d24-332c-4e29-ad08-579919b65509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:58.828785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f44d24-332c-4e29-ad08-579919b65509 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:58.844306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 940/1897] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:51:58.849666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f44d24-332c-4e29-ad08-579919b65509 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:58.862826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f44d24-332c-4e29-ad08-579919b65509 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:58.866356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a9998-4c6d-4bc1-87dd-7dc69e46342e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:58.867202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9998-4c6d-4bc1-87dd-7dc69e46342e admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:58.867906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9998-4c6d-4bc1-87dd-7dc69e46342e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:58.868200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9998-4c6d-4bc1-87dd-7dc69e46342e admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:58.868854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a9998-4c6d-4bc1-87dd-7dc69e46342e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:58.871161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a9998-4c6d-4bc1-87dd-7dc69e46342e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:58.901841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 958/1898] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:58 2022] POST /identity/v3/projects => generated 409 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:58.905776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b145d-5fdb-4654-aad8-91134429fb31 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:58.998963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b145d-5fdb-4654-aad8-91134429fb31 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:59.007094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8393ca-94ab-4ba1-82d3-a342310732f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:59.007851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 941/1899] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:58 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:51:59.008334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8393ca-94ab-4ba1-82d3-a342310732f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.008334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8393ca-94ab-4ba1-82d3-a342310732f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.008563 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8393ca-94ab-4ba1-82d3-a342310732f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.009617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8393ca-94ab-4ba1-82d3-a342310732f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:59.012644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8393ca-94ab-4ba1-82d3-a342310732f9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:59.013872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a9998-4c6d-4bc1-87dd-7dc69e46342e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:59.104935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a9998-4c6d-4bc1-87dd-7dc69e46342e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:59.111853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc2d4a-1ab5-47bc-b5cd-9f862c826705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:59.112303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc2d4a-1ab5-47bc-b5cd-9f862c826705 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.112594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc2d4a-1ab5-47bc-b5cd-9f862c826705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.113684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc2d4a-1ab5-47bc-b5cd-9f862c826705 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.114643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc2d4a-1ab5-47bc-b5cd-9f862c826705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:59.116671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc2d4a-1ab5-47bc-b5cd-9f862c826705 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:59.138944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 959/1900] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:58 2022] POST /identity/v3/users => generated 305 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:59.138944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8393ca-94ab-4ba1-82d3-a342310732f9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:59.164347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8393ca-94ab-4ba1-82d3-a342310732f9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:59.179159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38a6ae-2759-4c5e-9b22-4e01a1337fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:59.179783 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a6ae-2759-4c5e-9b22-4e01a1337fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.180063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a6ae-2759-4c5e-9b22-4e01a1337fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.180338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a6ae-2759-4c5e-9b22-4e01a1337fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.181046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38a6ae-2759-4c5e-9b22-4e01a1337fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:59.182723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38a6ae-2759-4c5e-9b22-4e01a1337fd3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:59.207743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 960/1901] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:59.208920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38a6ae-2759-4c5e-9b22-4e01a1337fd3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:59.234699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38a6ae-2759-4c5e-9b22-4e01a1337fd3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:59.247099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 942/1902] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:59 2022] POST /identity/v3/users => generated 284 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:59.251950 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc2d4a-1ab5-47bc-b5cd-9f862c826705 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:59.253545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7321d-7cd9-455d-b4b3-924ed8987d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:59.254619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7321d-7cd9-455d-b4b3-924ed8987d3a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.254989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7321d-7cd9-455d-b4b3-924ed8987d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.255266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7321d-7cd9-455d-b4b3-924ed8987d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.255921 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7321d-7cd9-455d-b4b3-924ed8987d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:59.257559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7321d-7cd9-455d-b4b3-924ed8987d3a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:59.276071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc2d4a-1ab5-47bc-b5cd-9f862c826705 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:59.286034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 961/1903] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:51:59.290090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7321d-7cd9-455d-b4b3-924ed8987d3a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:59.290616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef492098-5f03-499a-8434-dd8cac899e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:59.292135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef492098-5f03-499a-8434-dd8cac899e74 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.292429 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef492098-5f03-499a-8434-dd8cac899e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.292709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef492098-5f03-499a-8434-dd8cac899e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.293444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef492098-5f03-499a-8434-dd8cac899e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:59.295739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef492098-5f03-499a-8434-dd8cac899e74 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:59.311603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:59.316593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 943/1904] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:51:59.321005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef492098-5f03-499a-8434-dd8cac899e74 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:59.328694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7321d-7cd9-455d-b4b3-924ed8987d3a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:59.342452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88cec2-6877-47ff-b534-0358b2e6f6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:59.343193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cec2-6877-47ff-b534-0358b2e6f6bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.343476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cec2-6877-47ff-b534-0358b2e6f6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.343752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cec2-6877-47ff-b534-0358b2e6f6bf admin admin] PATH_INFO: `/v3/projects/53b0d9fad85448439d2d8f4d1f0c17d3/users/ff0fd6ee20fd40c18a4c4823131f92c5/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.344373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef492098-5f03-499a-8434-dd8cac899e74 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:59.361970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185522f-627e-46b7-8bc2-9f00e9e610f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:59.361970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185522f-627e-46b7-8bc2-9f00e9e610f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.361970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185522f-627e-46b7-8bc2-9f00e9e610f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.361970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185522f-627e-46b7-8bc2-9f00e9e610f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.362655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185522f-627e-46b7-8bc2-9f00e9e610f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:59.362655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185522f-627e-46b7-8bc2-9f00e9e610f7 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:59.375146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88cec2-6877-47ff-b534-0358b2e6f6bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b0d9fad85448439d2d8f4d1f0c17d3, user_id=ff0fd6ee20fd40c18a4c4823131f92c5, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:59.378464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88cec2-6877-47ff-b534-0358b2e6f6bf admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:59.381155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 944/1905] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:51:59.385487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185522f-627e-46b7-8bc2-9f00e9e610f7 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:59.398879 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:59.404291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185522f-627e-46b7-8bc2-9f00e9e610f7 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:59.414114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81b729-f5e6-41c9-9ba1-bf7082a24b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:59.414969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b729-f5e6-41c9-9ba1-bf7082a24b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.415443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b729-f5e6-41c9-9ba1-bf7082a24b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.415940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b729-f5e6-41c9-9ba1-bf7082a24b3b admin admin] PATH_INFO: `/v3/projects/d5672ba37e774169a41081e087e583ce/users/8765eff5c8974a52a03f594e815ee819/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.433125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81b729-f5e6-41c9-9ba1-bf7082a24b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5672ba37e774169a41081e087e583ce, user_id=8765eff5c8974a52a03f594e815ee819, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:59.436227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81b729-f5e6-41c9-9ba1-bf7082a24b3b admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:59.478921 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 962/1906] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:59 2022] PUT /identity/v3/projects/53b0d9fad85448439d2d8f4d1f0c17d3/users/ff0fd6ee20fd40c18a4c4823131f92c5/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:51:59.480618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ec5d9-38b0-41b8-a025-abefcacf7acf None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.480980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ec5d9-38b0-41b8-a025-abefcacf7acf None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.481343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ec5d9-38b0-41b8-a025-abefcacf7acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.517609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-207ec5d9-38b0-41b8-a025-abefcacf7acf None None] MFA Rules not processed for user `ff0fd6ee20fd40c18a4c4823131f92c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:59.525077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-207ec5d9-38b0-41b8-a025-abefcacf7acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:59.537792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 945/1907] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:51:59 2022] PUT /identity/v3/projects/d5672ba37e774169a41081e087e583ce/users/8765eff5c8974a52a03f594e815ee819/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:51:59.540412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cafeb-39c7-4721-83fa-828bbd2690eb None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.540817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cafeb-39c7-4721-83fa-828bbd2690eb None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.541196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cafeb-39c7-4721-83fa-828bbd2690eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.588447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-467cafeb-39c7-4721-83fa-828bbd2690eb None None] MFA Rules not processed for user `8765eff5c8974a52a03f594e815ee819`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:51:59.588447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-467cafeb-39c7-4721-83fa-828bbd2690eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:51:59.643773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 963/1908] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:59 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:59.648964 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ec5d9-38b0-41b8-a025-abefcacf7acf None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:59.675890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:59.697225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 946/1909] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:51:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:51:59.699915 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cafeb-39c7-4721-83fa-828bbd2690eb None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:59.745587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:51:59.774327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ec5d9-38b0-41b8-a025-abefcacf7acf None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:59.789860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d76f4-5dc2-4aaf-8248-ed199ccfb8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:59.789860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d76f4-5dc2-4aaf-8248-ed199ccfb8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.789860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d76f4-5dc2-4aaf-8248-ed199ccfb8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.789860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d76f4-5dc2-4aaf-8248-ed199ccfb8f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.790245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d76f4-5dc2-4aaf-8248-ed199ccfb8f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:59.792100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d76f4-5dc2-4aaf-8248-ed199ccfb8f2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:59.838999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cafeb-39c7-4721-83fa-828bbd2690eb None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:51:59.853249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccab60-dbda-43b3-91b3-eddaf295e662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:51:59.853932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccab60-dbda-43b3-91b3-eddaf295e662 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:51:59.854216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccab60-dbda-43b3-91b3-eddaf295e662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:51:59.854518 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccab60-dbda-43b3-91b3-eddaf295e662 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:51:59.858109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccab60-dbda-43b3-91b3-eddaf295e662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:51:59.860284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccab60-dbda-43b3-91b3-eddaf295e662 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:51:59.906335 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 964/1910] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:59 2022] POST /identity/v3/projects => generated 407 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:59.910649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d76f4-5dc2-4aaf-8248-ed199ccfb8f2 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:51:59.973444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 947/1911] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:51:59 2022] POST /identity/v3/projects => generated 369 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:51:59.976345 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccab60-dbda-43b3-91b3-eddaf295e662 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:00.055491 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d76f4-5dc2-4aaf-8248-ed199ccfb8f2 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:00.095311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccab60-dbda-43b3-91b3-eddaf295e662 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:00.116799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf11de-debe-434f-a766-ceeccfed2c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:00.117878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf11de-debe-434f-a766-ceeccfed2c39 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.119699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf11de-debe-434f-a766-ceeccfed2c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.120055 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf11de-debe-434f-a766-ceeccfed2c39 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.120811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf11de-debe-434f-a766-ceeccfed2c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:00.127130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf11de-debe-434f-a766-ceeccfed2c39 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:00.161842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89075cf7-ba08-4e9d-9ea5-d6dd022ddb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:00.162708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89075cf7-ba08-4e9d-9ea5-d6dd022ddb00 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.162992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89075cf7-ba08-4e9d-9ea5-d6dd022ddb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.163263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89075cf7-ba08-4e9d-9ea5-d6dd022ddb00 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.163909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89075cf7-ba08-4e9d-9ea5-d6dd022ddb00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:00.165899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89075cf7-ba08-4e9d-9ea5-d6dd022ddb00 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:00.242009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 948/1912] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:59 2022] POST /identity/v3/users => generated 291 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:00.251545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf11de-debe-434f-a766-ceeccfed2c39 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:00.267292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 965/1913] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:51:59 2022] POST /identity/v3/users => generated 310 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:52:00.270713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf11de-debe-434f-a766-ceeccfed2c39 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:00.271453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89075cf7-ba08-4e9d-9ea5-d6dd022ddb00 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:00.280437 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025ffdc-f02b-4b7d-9531-ff68feaf90a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:00.281390 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ffdc-f02b-4b7d-9531-ff68feaf90a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.281828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ffdc-f02b-4b7d-9531-ff68feaf90a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.282294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ffdc-f02b-4b7d-9531-ff68feaf90a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.283131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025ffdc-f02b-4b7d-9531-ff68feaf90a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:00.290443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025ffdc-f02b-4b7d-9531-ff68feaf90a6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:00.292931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89075cf7-ba08-4e9d-9ea5-d6dd022ddb00 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:00.307041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23ea98-e85f-4d67-94ba-eb29342888d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:00.307670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23ea98-e85f-4d67-94ba-eb29342888d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.308130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23ea98-e85f-4d67-94ba-eb29342888d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.308419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23ea98-e85f-4d67-94ba-eb29342888d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.312930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23ea98-e85f-4d67-94ba-eb29342888d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:00.312930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23ea98-e85f-4d67-94ba-eb29342888d1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:00.322181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 949/1914] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:00.327513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025ffdc-f02b-4b7d-9531-ff68feaf90a6 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:00.349068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 966/1915] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:52:00.352631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025ffdc-f02b-4b7d-9531-ff68feaf90a6 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:00.358197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23ea98-e85f-4d67-94ba-eb29342888d1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:00.360017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a5c41-a0a5-45e4-a06a-a16617d09225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:00.360923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a5c41-a0a5-45e4-a06a-a16617d09225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.361215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a5c41-a0a5-45e4-a06a-a16617d09225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.361490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a5c41-a0a5-45e4-a06a-a16617d09225 admin admin] PATH_INFO: `/v3/projects/e936e0874d1c46aab4bfa0f3caf1e3c8/users/08702bc762534bdcbaff343be90b89c7/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.377095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23ea98-e85f-4d67-94ba-eb29342888d1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:00.389823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a5c41-a0a5-45e4-a06a-a16617d09225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e936e0874d1c46aab4bfa0f3caf1e3c8, user_id=08702bc762534bdcbaff343be90b89c7, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:00.394194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47f51b-c4b1-4b5f-bec9-6a4714a743f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:00.394997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47f51b-c4b1-4b5f-bec9-6a4714a743f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.395933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47f51b-c4b1-4b5f-bec9-6a4714a743f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.396220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47f51b-c4b1-4b5f-bec9-6a4714a743f3 admin admin] PATH_INFO: `/v3/projects/be2f012db59b4e1c9c1aad3d38ee0e23/users/e9ad1234c00046149c1774a0a0953a18/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.397611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a5c41-a0a5-45e4-a06a-a16617d09225 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:00.424026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47f51b-c4b1-4b5f-bec9-6a4714a743f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2f012db59b4e1c9c1aad3d38ee0e23, user_id=e9ad1234c00046149c1774a0a0953a18, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:00.427913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47f51b-c4b1-4b5f-bec9-6a4714a743f3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:00.527349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 950/1916] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:00 2022] PUT /identity/v3/projects/e936e0874d1c46aab4bfa0f3caf1e3c8/users/08702bc762534bdcbaff343be90b89c7/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:00.532358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a5c41-a0a5-45e4-a06a-a16617d09225 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:00.556358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 967/1917] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:00 2022] PUT /identity/v3/projects/be2f012db59b4e1c9c1aad3d38ee0e23/users/e9ad1234c00046149c1774a0a0953a18/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:00.560201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47f51b-c4b1-4b5f-bec9-6a4714a743f3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:00.585082 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:00.634346 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:00.669506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a5c41-a0a5-45e4-a06a-a16617d09225 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:00.672829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47f51b-c4b1-4b5f-bec9-6a4714a743f3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:00.677109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f8d44-7151-4f5b-90d7-3f4855c78886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:00.677914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f8d44-7151-4f5b-90d7-3f4855c78886 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.678571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f8d44-7151-4f5b-90d7-3f4855c78886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.683228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f8d44-7151-4f5b-90d7-3f4855c78886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.683390 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283bba6-bda1-4fa9-82ae-6f72bfea4cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:00.683514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f8d44-7151-4f5b-90d7-3f4855c78886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:00.683605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283bba6-bda1-4fa9-82ae-6f72bfea4cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.683605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283bba6-bda1-4fa9-82ae-6f72bfea4cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.683605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283bba6-bda1-4fa9-82ae-6f72bfea4cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.683605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283bba6-bda1-4fa9-82ae-6f72bfea4cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:00.683908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f8d44-7151-4f5b-90d7-3f4855c78886 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:00.683991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283bba6-bda1-4fa9-82ae-6f72bfea4cb4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:00.697915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 968/1918] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:00.699049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283bba6-bda1-4fa9-82ae-6f72bfea4cb4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:00.706536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 951/1919] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:00.707673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f8d44-7151-4f5b-90d7-3f4855c78886 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:00.712022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283bba6-bda1-4fa9-82ae-6f72bfea4cb4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:00.717368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bb901-a021-4701-a5ae-d649670aecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:00.718297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bb901-a021-4701-a5ae-d649670aecce admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.718613 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bb901-a021-4701-a5ae-d649670aecce admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.718887 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bb901-a021-4701-a5ae-d649670aecce admin admin] PATH_INFO: `/v3/projects/be2f012db59b4e1c9c1aad3d38ee0e23/users/e9ad1234c00046149c1774a0a0953a18/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.721342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bb901-a021-4701-a5ae-d649670aecce admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2f012db59b4e1c9c1aad3d38ee0e23, user_id=e9ad1234c00046149c1774a0a0953a18, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:00.723298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f8d44-7151-4f5b-90d7-3f4855c78886 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:00.724748 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bb901-a021-4701-a5ae-d649670aecce admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:00.729418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df4868-ceb4-4ddd-94e7-4a0e2eb90af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:00.730089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df4868-ceb4-4ddd-94e7-4a0e2eb90af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.730366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df4868-ceb4-4ddd-94e7-4a0e2eb90af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.730711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df4868-ceb4-4ddd-94e7-4a0e2eb90af3 admin admin] PATH_INFO: `/v3/projects/e936e0874d1c46aab4bfa0f3caf1e3c8/users/08702bc762534bdcbaff343be90b89c7/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.732794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df4868-ceb4-4ddd-94e7-4a0e2eb90af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e936e0874d1c46aab4bfa0f3caf1e3c8, user_id=08702bc762534bdcbaff343be90b89c7, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:00.735879 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df4868-ceb4-4ddd-94e7-4a0e2eb90af3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:00.743072 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 969/1920] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:00 2022] PUT /identity/v3/projects/be2f012db59b4e1c9c1aad3d38ee0e23/users/e9ad1234c00046149c1774a0a0953a18/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:00.747915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6c133-ab48-4a88-b13d-78861cf61088 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.748281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 952/1921] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:00 2022] PUT /identity/v3/projects/e936e0874d1c46aab4bfa0f3caf1e3c8/users/08702bc762534bdcbaff343be90b89c7/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:00.748708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6c133-ab48-4a88-b13d-78861cf61088 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.750498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6c133-ab48-4a88-b13d-78861cf61088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.764394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e02c8f-a8da-4a60-adb8-626c0f9108a5 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.764394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e02c8f-a8da-4a60-adb8-626c0f9108a5 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.764394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e02c8f-a8da-4a60-adb8-626c0f9108a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:00.793322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8cf6c133-ab48-4a88-b13d-78861cf61088 None None] MFA Rules not processed for user `e9ad1234c00046149c1774a0a0953a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:00.794872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8cf6c133-ab48-4a88-b13d-78861cf61088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:00.829135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-e6e02c8f-a8da-4a60-adb8-626c0f9108a5 None None] MFA Rules not processed for user `08702bc762534bdcbaff343be90b89c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:00.831145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e6e02c8f-a8da-4a60-adb8-626c0f9108a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:00.892772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 970/1922] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:00.910378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6c133-ab48-4a88-b13d-78861cf61088 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:00.911979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8cf6c133-ab48-4a88-b13d-78861cf61088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:00.943185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 953/1923] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:52:00.947470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e02c8f-a8da-4a60-adb8-626c0f9108a5 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:00.947470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e6e02c8f-a8da-4a60-adb8-626c0f9108a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:00.992419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6c133-ab48-4a88-b13d-78861cf61088 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:00.996900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7470a-cc84-4b48-9805-cca724e8ff1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:00.997682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7470a-cc84-4b48-9805-cca724e8ff1e service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:00.997963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7470a-cc84-4b48-9805-cca724e8ff1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:00.998233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7470a-cc84-4b48-9805-cca724e8ff1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:01.029061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7470a-cc84-4b48-9805-cca724e8ff1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:01.030824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7470a-cc84-4b48-9805-cca724e8ff1e service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:01.063067 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e02c8f-a8da-4a60-adb8-626c0f9108a5 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:01.066969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 971/1924] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:52:01.072356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f40a5-967d-484a-acfa-5fa0c66d1ad0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:01.073492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f40a5-967d-484a-acfa-5fa0c66d1ad0 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:01.073973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f40a5-967d-484a-acfa-5fa0c66d1ad0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:01.074835 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f40a5-967d-484a-acfa-5fa0c66d1ad0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:01.092734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f40a5-967d-484a-acfa-5fa0c66d1ad0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:01.095636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f40a5-967d-484a-acfa-5fa0c66d1ad0 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:01.126739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 954/1925] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:01.251065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7470a-cc84-4b48-9805-cca724e8ff1e service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:01.262253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7470a-cc84-4b48-9805-cca724e8ff1e service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:01.266270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59663b06-84cc-4bb2-8a38-3f597b03ef14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:01.266927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59663b06-84cc-4bb2-8a38-3f597b03ef14 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:01.267208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59663b06-84cc-4bb2-8a38-3f597b03ef14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:01.267473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59663b06-84cc-4bb2-8a38-3f597b03ef14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:01.314875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:01.336400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59663b06-84cc-4bb2-8a38-3f597b03ef14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:01.343021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59663b06-84cc-4bb2-8a38-3f597b03ef14 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:01.349898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 972/1926] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:52:01.626935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f40a5-967d-484a-acfa-5fa0c66d1ad0 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:01.642766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f40a5-967d-484a-acfa-5fa0c66d1ad0 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:01.655822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a392-fc57-4541-b9bd-09f429620a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:01.656459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a392-fc57-4541-b9bd-09f429620a7e service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:01.656793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a392-fc57-4541-b9bd-09f429620a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:01.657061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a392-fc57-4541-b9bd-09f429620a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:01.714350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:01.800133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98a392-fc57-4541-b9bd-09f429620a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:01.803752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98a392-fc57-4541-b9bd-09f429620a7e service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:01.825409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 955/1927] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:01 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:05.936761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59663b06-84cc-4bb2-8a38-3f597b03ef14 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:05.939562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-59663b06-84cc-4bb2-8a38-3f597b03ef14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:06.088737 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59663b06-84cc-4bb2-8a38-3f597b03ef14 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:06.100121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:06.101621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:06.118688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:06.119184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] PATH_INFO: `/v3/users/46e697e14f5a4d6288e358b7365c9498` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:06.144259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e697e14f5a4d6288e358b7365c9498)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:06.148183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:06.408331 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e697e14f5a4d6288e358b7365c9498'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:06.439867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e697e14f5a4d6288e358b7365c9498'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:06.450094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e697e14f5a4d6288e358b7365c9498'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:06.521773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e697e14f5a4d6288e358b7365c9498'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:06.527261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e697e14f5a4d6288e358b7365c9498'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:06.537714 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 973/1928] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:05 2022] DELETE /identity/v3/users/46e697e14f5a4d6288e358b7365c9498 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:06.543260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eb83d-5966-4a64-95e7-c6d0c030fc88 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:06.543260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eb83d-5966-4a64-95e7-c6d0c030fc88 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:06.543260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eb83d-5966-4a64-95e7-c6d0c030fc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:06.566483 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-c52eb83d-5966-4a64-95e7-c6d0c030fc88 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:06.566483 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c52eb83d-5966-4a64-95e7-c6d0c030fc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:06.654405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 956/1929] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:52:06.668569 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:06.670627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:06.766724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed5f98-d2c7-45ec-8258-20fdb22ca600 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:06.772263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab1da7-b78e-4a01-9a9d-4ba95af6ed60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:06.773513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab1da7-b78e-4a01-9a9d-4ba95af6ed60 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:06.774507 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab1da7-b78e-4a01-9a9d-4ba95af6ed60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:06.775085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab1da7-b78e-4a01-9a9d-4ba95af6ed60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:06.796734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab1da7-b78e-4a01-9a9d-4ba95af6ed60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:06.799740 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab1da7-b78e-4a01-9a9d-4ba95af6ed60 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:06.808475 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:06.821068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 974/1930] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:07.285155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52eb83d-5966-4a64-95e7-c6d0c030fc88 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:07.289952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c52eb83d-5966-4a64-95e7-c6d0c030fc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:07.327015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52eb83d-5966-4a64-95e7-c6d0c030fc88 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:07.334346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:07.334809 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:07.335152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:07.335436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] PATH_INFO: `/v3/projects/7524ddcc684c4afe8fac71722c6f59ba` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:07.336662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7524ddcc684c4afe8fac71722c6f59ba)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:07.338713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:07.505518 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7524ddcc684c4afe8fac71722c6f59ba', 'request_id': 'req-fc60324f-e3ad-4596-90e9-d736375d58f7'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:07.618478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7524ddcc684c4afe8fac71722c6f59ba', 'request_id': 'req-fc60324f-e3ad-4596-90e9-d736375d58f7'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:07.626975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7524ddcc684c4afe8fac71722c6f59ba', 'request_id': 'req-fc60324f-e3ad-4596-90e9-d736375d58f7'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:07.653180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] The token cache is being invalidate because project 7524ddcc684c4afe8fac71722c6f59ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:07.653459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:07.654843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:07.657251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 957/1931] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:07 2022] DELETE /identity/v3/projects/7524ddcc684c4afe8fac71722c6f59ba => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:07.661583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab1da7-b78e-4a01-9a9d-4ba95af6ed60 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:07.686102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab1da7-b78e-4a01-9a9d-4ba95af6ed60 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:07.813007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:07.813709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:07.813990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:07.814290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] PATH_INFO: `/v3/users/79f3c10ccf8b4f4bb10f506ef6af3ada` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:07.815673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79f3c10ccf8b4f4bb10f506ef6af3ada)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:07.817681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:08.079309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79f3c10ccf8b4f4bb10f506ef6af3ada'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:08.095550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79f3c10ccf8b4f4bb10f506ef6af3ada'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:08.106684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79f3c10ccf8b4f4bb10f506ef6af3ada'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:08.218763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79f3c10ccf8b4f4bb10f506ef6af3ada'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:08.219400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79f3c10ccf8b4f4bb10f506ef6af3ada'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:08.226844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 975/1932] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:07 2022] DELETE /identity/v3/users/79f3c10ccf8b4f4bb10f506ef6af3ada => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:08.826086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:08.827600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:08.852487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60324f-e3ad-4596-90e9-d736375d58f7 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:09.009872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:09.044951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:09.045559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:09.046314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:09.046314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:09.046314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] PATH_INFO: `/v3/projects/65d89c9e4d1a417f86fd541ca254ea9f` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:09.047727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d89c9e4d1a417f86fd541ca254ea9f)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:09.049234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:09.050248 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:09.074813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0048b3-d021-4cf3-8d19-385f1690d295 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:09.084203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cc3c4-64ab-44e3-9570-0a39f8835ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:09.085986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cc3c4-64ab-44e3-9570-0a39f8835ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:09.086509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cc3c4-64ab-44e3-9570-0a39f8835ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:09.086974 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cc3c4-64ab-44e3-9570-0a39f8835ba0 admin admin] PATH_INFO: `/v3/users/8765eff5c8974a52a03f594e815ee819` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:09.088896 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cc3c4-64ab-44e3-9570-0a39f8835ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8765eff5c8974a52a03f594e815ee819)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:09.091815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cc3c4-64ab-44e3-9570-0a39f8835ba0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:09.189151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d89c9e4d1a417f86fd541ca254ea9f', 'request_id': 'req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:09.317516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d89c9e4d1a417f86fd541ca254ea9f', 'request_id': 'req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:09.328661 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d89c9e4d1a417f86fd541ca254ea9f', 'request_id': 'req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:09.360227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] The token cache is being invalidate because project 65d89c9e4d1a417f86fd541ca254ea9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:09.360605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:09.362492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:09.366312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 958/1933] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:08 2022] DELETE /identity/v3/projects/65d89c9e4d1a417f86fd541ca254ea9f => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:09.370122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:09.371883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:09.374457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b10cc3c4-64ab-44e3-9570-0a39f8835ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8765eff5c8974a52a03f594e815ee819'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:09.414222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b10cc3c4-64ab-44e3-9570-0a39f8835ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8765eff5c8974a52a03f594e815ee819'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:09.457412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b10cc3c4-64ab-44e3-9570-0a39f8835ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8765eff5c8974a52a03f594e815ee819'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:09.459864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:09.520972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b10cc3c4-64ab-44e3-9570-0a39f8835ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8765eff5c8974a52a03f594e815ee819'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:09.522959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b10cc3c4-64ab-44e3-9570-0a39f8835ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8765eff5c8974a52a03f594e815ee819'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:09.532626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 976/1934] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:09 2022] DELETE /identity/v3/users/8765eff5c8974a52a03f594e815ee819 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:09.537374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecf5d7-e2af-4f88-9948-395abfe6776c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:09.537873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecf5d7-e2af-4f88-9948-395abfe6776c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:09.538244 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecf5d7-e2af-4f88-9948-395abfe6776c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:09.550876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bb28e-1644-49cf-9365-57f8ae3a8c5a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:09.566827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a0ecf5d7-e2af-4f88-9948-395abfe6776c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:09.569586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a0ecf5d7-e2af-4f88-9948-395abfe6776c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:09.678025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:09.679357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:09.679920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:09.680514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] PATH_INFO: `/v3/users/de761a9e0a6c4b74a73960c431c2c4b4` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:09.686328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de761a9e0a6c4b74a73960c431c2c4b4)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:09.692636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:09.748447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 977/1935] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:09.790486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecf5d7-e2af-4f88-9948-395abfe6776c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:09.791968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a0ecf5d7-e2af-4f88-9948-395abfe6776c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:09.820750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:09.950472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de761a9e0a6c4b74a73960c431c2c4b4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:09.984672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecf5d7-e2af-4f88-9948-395abfe6776c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:09.996873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de761a9e0a6c4b74a73960c431c2c4b4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:10.001568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d23db1-5a52-4c6c-b276-9d2c24c008fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:10.004984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d23db1-5a52-4c6c-b276-9d2c24c008fe service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:10.007600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d23db1-5a52-4c6c-b276-9d2c24c008fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:10.008249 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d23db1-5a52-4c6c-b276-9d2c24c008fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:10.015776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de761a9e0a6c4b74a73960c431c2c4b4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:10.030704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d23db1-5a52-4c6c-b276-9d2c24c008fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:10.034995 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d23db1-5a52-4c6c-b276-9d2c24c008fe service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:10.057695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 978/1936] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:10.156244 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de761a9e0a6c4b74a73960c431c2c4b4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:10.156924 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de761a9e0a6c4b74a73960c431c2c4b4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:10.167623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 959/1937] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:09 2022] DELETE /identity/v3/users/de761a9e0a6c4b74a73960c431c2c4b4 => generated 0 bytes in 799 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:11.124151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d23db1-5a52-4c6c-b276-9d2c24c008fe service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:11.126638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:11.129387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-88d23db1-5a52-4c6c-b276-9d2c24c008fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:11.130003 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:11.165274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d23db1-5a52-4c6c-b276-9d2c24c008fe service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:11.337369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b001518-d155-4715-aaf3-0b9233adea59 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:11.365568 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:11.367855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:11.371435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:11.373156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] PATH_INFO: `/v3/projects/0c64bb8c630c4067adfe31c69d59b8be` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:11.377977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c64bb8c630c4067adfe31c69d59b8be)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:11.385469 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:11.433773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:11.436060 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:11.438115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:11.442443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] PATH_INFO: `/v3/projects/d5672ba37e774169a41081e087e583ce` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:11.452433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5672ba37e774169a41081e087e583ce)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:11.460959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:11.637007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c64bb8c630c4067adfe31c69d59b8be', 'request_id': 'req-94ea9256-97e2-4336-bff5-2e686dcdeb2f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:11.732283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=1,keystone:DELETE=4,keystone:SELECT=95 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:11.745997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c64bb8c630c4067adfe31c69d59b8be', 'request_id': 'req-94ea9256-97e2-4336-bff5-2e686dcdeb2f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:11.756167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5672ba37e774169a41081e087e583ce', 'request_id': 'req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:11.767284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c64bb8c630c4067adfe31c69d59b8be', 'request_id': 'req-94ea9256-97e2-4336-bff5-2e686dcdeb2f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:11.770346 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5672ba37e774169a41081e087e583ce', 'request_id': 'req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:11.791513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] The token cache is being invalidate because project 0c64bb8c630c4067adfe31c69d59b8be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:11.791789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:11.793198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-94ea9256-97e2-4336-bff5-2e686dcdeb2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:11.794408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 960/1938] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:11 2022] DELETE /identity/v3/projects/0c64bb8c630c4067adfe31c69d59b8be => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:11.801300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff9e1-2ca0-4a31-bf0c-5f2b4d70460e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:11.802125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff9e1-2ca0-4a31-bf0c-5f2b4d70460e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:11.802125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff9e1-2ca0-4a31-bf0c-5f2b4d70460e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:11.847401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-9b9ff9e1-2ca0-4a31-bf0c-5f2b4d70460e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:11.851200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9b9ff9e1-2ca0-4a31-bf0c-5f2b4d70460e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:11.858127 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5672ba37e774169a41081e087e583ce', 'request_id': 'req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:11.883353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] The token cache is being invalidate because project d5672ba37e774169a41081e087e583ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:11.883968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:11.885095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:11.887955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 979/1939] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:11 2022] DELETE /identity/v3/projects/d5672ba37e774169a41081e087e583ce => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:11.895373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:11.903785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:11.921362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:11.975374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 961/1940] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:52:11.985074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c2ee1-7120-4be9-938f-3846c396f6ee None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:11.986046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c2ee1-7120-4be9-938f-3846c396f6ee None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:11.987218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c2ee1-7120-4be9-938f-3846c396f6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:11.992713 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5e45e-ce8d-454a-a9af-ec24fcc1a71f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:11.999218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:12.000063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:12.000449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:12.000841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] PATH_INFO: `/v3/users/08702bc762534bdcbaff343be90b89c7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:12.002784 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08702bc762534bdcbaff343be90b89c7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:12.005729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:12.025523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-658c2ee1-7120-4be9-938f-3846c396f6ee None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:12.028080 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-658c2ee1-7120-4be9-938f-3846c396f6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:12.052184 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 962/1941] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:12.060821 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c2ee1-7120-4be9-938f-3846c396f6ee None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:12.079881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c2ee1-7120-4be9-938f-3846c396f6ee None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:12.091938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0da7f8-1b1e-4192-8bb9-f589fce9f777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:12.092594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0da7f8-1b1e-4192-8bb9-f589fce9f777 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:12.093433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0da7f8-1b1e-4192-8bb9-f589fce9f777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:12.093433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0da7f8-1b1e-4192-8bb9-f589fce9f777 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:12.093959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0da7f8-1b1e-4192-8bb9-f589fce9f777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:52:12.094274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0da7f8-1b1e-4192-8bb9-f589fce9f777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:12.098415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0da7f8-1b1e-4192-8bb9-f589fce9f777 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:12.121479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 963/1942] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:52:12.125728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0da7f8-1b1e-4192-8bb9-f589fce9f777 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:12.160683 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0da7f8-1b1e-4192-8bb9-f589fce9f777 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:12.160683 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7072a-ca52-4157-9eef-0bc297e591a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:12.160683 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7072a-ca52-4157-9eef-0bc297e591a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:12.160683 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7072a-ca52-4157-9eef-0bc297e591a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:12.161655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7072a-ca52-4157-9eef-0bc297e591a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:12.161655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7072a-ca52-4157-9eef-0bc297e591a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:12.161655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7072a-ca52-4157-9eef-0bc297e591a3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:12.337205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 964/1943] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:12 2022] POST /identity/v3/projects => generated 365 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:52:12.343812 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7072a-ca52-4157-9eef-0bc297e591a3 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:12.394449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08702bc762534bdcbaff343be90b89c7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:12.414269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08702bc762534bdcbaff343be90b89c7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:12.427375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08702bc762534bdcbaff343be90b89c7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:12.446552 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7072a-ca52-4157-9eef-0bc297e591a3 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:12.455258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc86c4-31d8-4f0c-88d4-acd410e92c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:12.457884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc86c4-31d8-4f0c-88d4-acd410e92c44 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:12.458265 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc86c4-31d8-4f0c-88d4-acd410e92c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:12.458661 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc86c4-31d8-4f0c-88d4-acd410e92c44 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:12.460782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc86c4-31d8-4f0c-88d4-acd410e92c44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:12.462070 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc86c4-31d8-4f0c-88d4-acd410e92c44 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:12.545224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08702bc762534bdcbaff343be90b89c7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:12.547025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08702bc762534bdcbaff343be90b89c7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:12.560153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 980/1944] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:11 2022] DELETE /identity/v3/users/08702bc762534bdcbaff343be90b89c7 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:12.586859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 965/1945] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:12 2022] POST /identity/v3/users => generated 283 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:52:12.596234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:12.613907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:12.746641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:12.797957 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53297434-2ce9-41ee-8487-840e814560a4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:12.806342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148b548-8c3b-4d18-b2c6-6e4f25e49de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:12.807503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b548-8c3b-4d18-b2c6-6e4f25e49de0 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:12.808037 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b548-8c3b-4d18-b2c6-6e4f25e49de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:12.808578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b548-8c3b-4d18-b2c6-6e4f25e49de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:12.809827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148b548-8c3b-4d18-b2c6-6e4f25e49de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:12.816383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148b548-8c3b-4d18-b2c6-6e4f25e49de0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:12.838359 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 981/1946] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:12.848710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc86c4-31d8-4f0c-88d4-acd410e92c44 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:12.861472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc86c4-31d8-4f0c-88d4-acd410e92c44 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:12.867872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19654d-0f46-44e9-a9d9-23368a11240b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:12.868523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19654d-0f46-44e9-a9d9-23368a11240b admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:12.868803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19654d-0f46-44e9-a9d9-23368a11240b admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:12.869202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19654d-0f46-44e9-a9d9-23368a11240b admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:12.869855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19654d-0f46-44e9-a9d9-23368a11240b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:12.871493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19654d-0f46-44e9-a9d9-23368a11240b admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:12.886553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 966/1947] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:12.892772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148b548-8c3b-4d18-b2c6-6e4f25e49de0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:12.910100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148b548-8c3b-4d18-b2c6-6e4f25e49de0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:12.916808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f816fb2-edd4-4465-bf54-4989557d68fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:12.917654 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f816fb2-edd4-4465-bf54-4989557d68fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:12.918085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f816fb2-edd4-4465-bf54-4989557d68fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:12.918590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f816fb2-edd4-4465-bf54-4989557d68fb admin admin] PATH_INFO: `/v3/projects/fca828c8bda9425a915a390768976f99/users/ca316d3889124ac2a16be6b86326d020/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:12.942381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f816fb2-edd4-4465-bf54-4989557d68fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca828c8bda9425a915a390768976f99, user_id=ca316d3889124ac2a16be6b86326d020, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:12.946105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f816fb2-edd4-4465-bf54-4989557d68fb admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:13.031504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 982/1948] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:12 2022] PUT /identity/v3/projects/fca828c8bda9425a915a390768976f99/users/ca316d3889124ac2a16be6b86326d020/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:13.057736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e02d12-cd16-4063-aaf4-092c05d900c4 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:13.058153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e02d12-cd16-4063-aaf4-092c05d900c4 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:13.059443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e02d12-cd16-4063-aaf4-092c05d900c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:13.148892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-49e02d12-cd16-4063-aaf4-092c05d900c4 None None] MFA Rules not processed for user `ca316d3889124ac2a16be6b86326d020`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:13.148892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-49e02d12-cd16-4063-aaf4-092c05d900c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:13.298599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:13.325659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 967/1949] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:13.348905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f816fb2-edd4-4465-bf54-4989557d68fb admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:13.359468 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2f816fb2-edd4-4465-bf54-4989557d68fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:13.382317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e02d12-cd16-4063-aaf4-092c05d900c4 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:13.384717 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-49e02d12-cd16-4063-aaf4-092c05d900c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:13.511426 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f816fb2-edd4-4465-bf54-4989557d68fb admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:13.527907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb30a6-2f3d-4694-a158-bed5ebbb858f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:13.528539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb30a6-2f3d-4694-a158-bed5ebbb858f service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:13.528885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb30a6-2f3d-4694-a158-bed5ebbb858f service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:13.529324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb30a6-2f3d-4694-a158-bed5ebbb858f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:13.561232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e02d12-cd16-4063-aaf4-092c05d900c4 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:13.561801 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb30a6-2f3d-4694-a158-bed5ebbb858f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:13.568559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb30a6-2f3d-4694-a158-bed5ebbb858f service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:13.568763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:13.569088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:13.569441 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:13.569728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] PATH_INFO: `/v3/projects/e936e0874d1c46aab4bfa0f3caf1e3c8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:13.571643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e936e0874d1c46aab4bfa0f3caf1e3c8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:13.574825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:13.591236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 983/1950] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:52:13.789935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e936e0874d1c46aab4bfa0f3caf1e3c8', 'request_id': 'req-5b4e201e-9e39-4ffd-b5f4-301ca544b952'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:13.858698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e936e0874d1c46aab4bfa0f3caf1e3c8', 'request_id': 'req-5b4e201e-9e39-4ffd-b5f4-301ca544b952'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:13.871595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e936e0874d1c46aab4bfa0f3caf1e3c8', 'request_id': 'req-5b4e201e-9e39-4ffd-b5f4-301ca544b952'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:13.892911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] The token cache is being invalidate because project e936e0874d1c46aab4bfa0f3caf1e3c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:13.894536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:13.894776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b4e201e-9e39-4ffd-b5f4-301ca544b952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:13.898838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 968/1951] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:13 2022] DELETE /identity/v3/projects/e936e0874d1c46aab4bfa0f3caf1e3c8 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:13.903537 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f79c8-2b20-47ce-a8f0-b118641e3d65 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:13.904054 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f79c8-2b20-47ce-a8f0-b118641e3d65 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:13.904514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f79c8-2b20-47ce-a8f0-b118641e3d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:13.932895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ab5f79c8-2b20-47ce-a8f0-b118641e3d65 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:13.934159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ab5f79c8-2b20-47ce-a8f0-b118641e3d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:14.036947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 984/1952] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:14.038893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f9448-b220-4ca2-a1ef-c950538d723a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:14.039182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f9448-b220-4ca2-a1ef-c950538d723a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:14.039471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f9448-b220-4ca2-a1ef-c950538d723a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:14.066652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-8d3f9448-b220-4ca2-a1ef-c950538d723a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:14.078675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8d3f9448-b220-4ca2-a1ef-c950538d723a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:14.097251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 969/1953] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:14.103043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f79c8-2b20-47ce-a8f0-b118641e3d65 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:14.119667 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f79c8-2b20-47ce-a8f0-b118641e3d65 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:14.127134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91323b-386b-42fb-b5fa-35eca3f0bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:14.128225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91323b-386b-42fb-b5fa-35eca3f0bd39 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:14.128613 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91323b-386b-42fb-b5fa-35eca3f0bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:14.128991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91323b-386b-42fb-b5fa-35eca3f0bd39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:14.129828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91323b-386b-42fb-b5fa-35eca3f0bd39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:52:14.130471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91323b-386b-42fb-b5fa-35eca3f0bd39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:14.133133 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91323b-386b-42fb-b5fa-35eca3f0bd39 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:14.162878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 985/1954] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:52:14.167616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f9448-b220-4ca2-a1ef-c950538d723a None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:14.192446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f9448-b220-4ca2-a1ef-c950538d723a None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:14.202012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16d831-9378-409f-808a-770ef036509a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:14.202824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16d831-9378-409f-808a-770ef036509a admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:14.203232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16d831-9378-409f-808a-770ef036509a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:14.207367 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16d831-9378-409f-808a-770ef036509a admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:14.207367 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16d831-9378-409f-808a-770ef036509a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:14.208543 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16d831-9378-409f-808a-770ef036509a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:14.332909 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 970/1955] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:14 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:14.339465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91323b-386b-42fb-b5fa-35eca3f0bd39 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:14.359847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91323b-386b-42fb-b5fa-35eca3f0bd39 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:14.449255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:14.485079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b112c-f58b-43a4-a9cc-23ed5f4dedb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:14.486404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b112c-f58b-43a4-a9cc-23ed5f4dedb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:14.486924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b112c-f58b-43a4-a9cc-23ed5f4dedb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:14.487435 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b112c-f58b-43a4-a9cc-23ed5f4dedb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:14.489068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b112c-f58b-43a4-a9cc-23ed5f4dedb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:14.493485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b112c-f58b-43a4-a9cc-23ed5f4dedb5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:14.641195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 986/1956] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:14 2022] POST /identity/v3/users => generated 292 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:14.648126 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16d831-9378-409f-808a-770ef036509a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:14.659776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16d831-9378-409f-808a-770ef036509a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:14.664677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f2b82-f68a-41a7-b2ca-aa7fdd2ff64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:14.665363 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f2b82-f68a-41a7-b2ca-aa7fdd2ff64c admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:14.665707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f2b82-f68a-41a7-b2ca-aa7fdd2ff64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:14.665988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f2b82-f68a-41a7-b2ca-aa7fdd2ff64c admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:14.666699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f2b82-f68a-41a7-b2ca-aa7fdd2ff64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:14.668892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f2b82-f68a-41a7-b2ca-aa7fdd2ff64c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:14.690937 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 971/1957] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:14.692320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b112c-f58b-43a4-a9cc-23ed5f4dedb5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:14.703930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b112c-f58b-43a4-a9cc-23ed5f4dedb5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:14.710291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562bc42-84ea-437c-8b34-264941a1c470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:14.711166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562bc42-84ea-437c-8b34-264941a1c470 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:14.711547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562bc42-84ea-437c-8b34-264941a1c470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:14.711875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562bc42-84ea-437c-8b34-264941a1c470 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:14.712670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562bc42-84ea-437c-8b34-264941a1c470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:14.714534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562bc42-84ea-437c-8b34-264941a1c470 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:14.733194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 987/1958] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:14.739271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f2b82-f68a-41a7-b2ca-aa7fdd2ff64c admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:14.755266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f2b82-f68a-41a7-b2ca-aa7fdd2ff64c admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:14.762461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cffc70-88c4-422e-a72b-63aa9ae2ca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:14.763046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cffc70-88c4-422e-a72b-63aa9ae2ca27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:14.763516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cffc70-88c4-422e-a72b-63aa9ae2ca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:14.763870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cffc70-88c4-422e-a72b-63aa9ae2ca27 admin admin] PATH_INFO: `/v3/projects/7e8059926afb458b96a7af4b2acacdf1/users/1c2ab9863f7141d4ab989822e9f1c133/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:14.789631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cffc70-88c4-422e-a72b-63aa9ae2ca27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8059926afb458b96a7af4b2acacdf1, user_id=1c2ab9863f7141d4ab989822e9f1c133, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:14.795733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cffc70-88c4-422e-a72b-63aa9ae2ca27 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:14.953176 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 972/1959] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:14 2022] PUT /identity/v3/projects/7e8059926afb458b96a7af4b2acacdf1/users/1c2ab9863f7141d4ab989822e9f1c133/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:14.975558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d10931-cb9b-492b-a66e-faf4c3d7052b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:14.976051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d10931-cb9b-492b-a66e-faf4c3d7052b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:14.976541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d10931-cb9b-492b-a66e-faf4c3d7052b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:15.028657 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a6d10931-cb9b-492b-a66e-faf4c3d7052b None None] MFA Rules not processed for user `1c2ab9863f7141d4ab989822e9f1c133`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:15.032582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a6d10931-cb9b-492b-a66e-faf4c3d7052b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:15.150650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 988/1960] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:52:15.156405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cffc70-88c4-422e-a72b-63aa9ae2ca27 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:15.220088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:15.264878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d10931-cb9b-492b-a66e-faf4c3d7052b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:15.267652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a6d10931-cb9b-492b-a66e-faf4c3d7052b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:15.282785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cffc70-88c4-422e-a72b-63aa9ae2ca27 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:15.289798 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ef002-8152-4c71-8b93-058e05b98d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:15.290090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d10931-cb9b-492b-a66e-faf4c3d7052b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:15.292331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ef002-8152-4c71-8b93-058e05b98d41 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:15.292824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ef002-8152-4c71-8b93-058e05b98d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:15.293201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ef002-8152-4c71-8b93-058e05b98d41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:15.294004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ef002-8152-4c71-8b93-058e05b98d41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:15.295549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44531b31-138e-49ba-b7af-3e4c05e08bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:15.296611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531b31-138e-49ba-b7af-3e4c05e08bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:15.296611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531b31-138e-49ba-b7af-3e4c05e08bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:15.296868 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ef002-8152-4c71-8b93-058e05b98d41 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:15.297033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531b31-138e-49ba-b7af-3e4c05e08bfd admin admin] PATH_INFO: `/v3/users/ff0fd6ee20fd40c18a4c4823131f92c5` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:15.298102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44531b31-138e-49ba-b7af-3e4c05e08bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff0fd6ee20fd40c18a4c4823131f92c5)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:15.300251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44531b31-138e-49ba-b7af-3e4c05e08bfd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:15.427653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 973/1961] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:15 2022] POST /identity/v3/projects => generated 381 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:15.437252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ef002-8152-4c71-8b93-058e05b98d41 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:15.560043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ef002-8152-4c71-8b93-058e05b98d41 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:15.566841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acdfe7-53fb-4971-ba03-40e7c6178b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:15.568335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acdfe7-53fb-4971-ba03-40e7c6178b4f admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:15.568621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acdfe7-53fb-4971-ba03-40e7c6178b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:15.568900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acdfe7-53fb-4971-ba03-40e7c6178b4f admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:15.569574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acdfe7-53fb-4971-ba03-40e7c6178b4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:15.572595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acdfe7-53fb-4971-ba03-40e7c6178b4f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:15.592339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-44531b31-138e-49ba-b7af-3e4c05e08bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff0fd6ee20fd40c18a4c4823131f92c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:15.608827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-44531b31-138e-49ba-b7af-3e4c05e08bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff0fd6ee20fd40c18a4c4823131f92c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:15.619583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-44531b31-138e-49ba-b7af-3e4c05e08bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff0fd6ee20fd40c18a4c4823131f92c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:15.684464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 974/1962] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:15 2022] POST /identity/v3/users => generated 297 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:15.691866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acdfe7-53fb-4971-ba03-40e7c6178b4f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:15.699920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-44531b31-138e-49ba-b7af-3e4c05e08bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff0fd6ee20fd40c18a4c4823131f92c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:15.701161 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-44531b31-138e-49ba-b7af-3e4c05e08bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff0fd6ee20fd40c18a4c4823131f92c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:15.719686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acdfe7-53fb-4971-ba03-40e7c6178b4f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:15.720890 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 989/1963] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:15 2022] DELETE /identity/v3/users/ff0fd6ee20fd40c18a4c4823131f92c5 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:15.726677 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbca893-e620-4779-a6f4-f0b8f62dff10 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:15.727150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbca893-e620-4779-a6f4-f0b8f62dff10 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:15.727534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbca893-e620-4779-a6f4-f0b8f62dff10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:15.786713 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6fbca893-e620-4779-a6f4-f0b8f62dff10 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:15.788909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6fbca893-e620-4779-a6f4-f0b8f62dff10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:15.860727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:15.872279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac42537-c5d8-4044-85c9-c630754985c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:15.878094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac42537-c5d8-4044-85c9-c630754985c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:15.878094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac42537-c5d8-4044-85c9-c630754985c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:15.878094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac42537-c5d8-4044-85c9-c630754985c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:15.878094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac42537-c5d8-4044-85c9-c630754985c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:15.878094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac42537-c5d8-4044-85c9-c630754985c4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:15.904183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 975/1964] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:15.913358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac42537-c5d8-4044-85c9-c630754985c4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:15.920188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-cac42537-c5d8-4044-85c9-c630754985c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:15.948867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac42537-c5d8-4044-85c9-c630754985c4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:15.954749 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baccab-e0bb-4a34-aea0-584ad0b58877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:15.955730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baccab-e0bb-4a34-aea0-584ad0b58877 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:15.956071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baccab-e0bb-4a34-aea0-584ad0b58877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:15.956350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baccab-e0bb-4a34-aea0-584ad0b58877 admin admin] PATH_INFO: `/v3/projects/1cf62861d9ad43a08f4c1ccb576e6d1b/users/39d85a67f679436c9015067427e3542e/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:15.977429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 990/1965] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:15.989069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbca893-e620-4779-a6f4-f0b8f62dff10 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:15.991603 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baccab-e0bb-4a34-aea0-584ad0b58877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf62861d9ad43a08f4c1ccb576e6d1b, user_id=39d85a67f679436c9015067427e3542e, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:15.992050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6fbca893-e620-4779-a6f4-f0b8f62dff10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:16.002691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baccab-e0bb-4a34-aea0-584ad0b58877 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:16.096744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 976/1966] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:15 2022] PUT /identity/v3/projects/1cf62861d9ad43a08f4c1ccb576e6d1b/users/39d85a67f679436c9015067427e3542e/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:16.098347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baccab-e0bb-4a34-aea0-584ad0b58877 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:16.132282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:16.161906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbca893-e620-4779-a6f4-f0b8f62dff10 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:16.295830 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baccab-e0bb-4a34-aea0-584ad0b58877 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:16.312952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bc31e-8754-4503-9178-484bb40652c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:16.314230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bc31e-8754-4503-9178-484bb40652c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:16.315154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bc31e-8754-4503-9178-484bb40652c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:16.315766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bc31e-8754-4503-9178-484bb40652c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:16.317133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bc31e-8754-4503-9178-484bb40652c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:16.322104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bc31e-8754-4503-9178-484bb40652c4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:16.346114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca36d91-ad10-48cb-9374-895c48d38876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:16.347070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca36d91-ad10-48cb-9374-895c48d38876 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:16.347533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca36d91-ad10-48cb-9374-895c48d38876 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:16.347961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca36d91-ad10-48cb-9374-895c48d38876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:16.348986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 977/1967] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:16.358559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bc31e-8754-4503-9178-484bb40652c4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:16.374867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca36d91-ad10-48cb-9374-895c48d38876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:16.376405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca36d91-ad10-48cb-9374-895c48d38876 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:16.387839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bc31e-8754-4503-9178-484bb40652c4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:16.388765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 991/1968] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:16.405406 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe22c39-b53e-4389-8901-b84d79bb28c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:16.407007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22c39-b53e-4389-8901-b84d79bb28c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:16.407549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22c39-b53e-4389-8901-b84d79bb28c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:16.407953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22c39-b53e-4389-8901-b84d79bb28c4 admin admin] PATH_INFO: `/v3/projects/1cf62861d9ad43a08f4c1ccb576e6d1b/users/39d85a67f679436c9015067427e3542e/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:16.416329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe22c39-b53e-4389-8901-b84d79bb28c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf62861d9ad43a08f4c1ccb576e6d1b, user_id=39d85a67f679436c9015067427e3542e, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:16.424340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe22c39-b53e-4389-8901-b84d79bb28c4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:16.479636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 978/1969] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:16 2022] PUT /identity/v3/projects/1cf62861d9ad43a08f4c1ccb576e6d1b/users/39d85a67f679436c9015067427e3542e/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:16.488584 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d4be1-e9e2-419f-a43a-985346b7be75 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:16.488891 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d4be1-e9e2-419f-a43a-985346b7be75 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:16.489184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d4be1-e9e2-419f-a43a-985346b7be75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:16.531201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-570d4be1-e9e2-419f-a43a-985346b7be75 None None] MFA Rules not processed for user `39d85a67f679436c9015067427e3542e`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:16.532666 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-570d4be1-e9e2-419f-a43a-985346b7be75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:16.677539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 992/1970] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:52:16.696095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe22c39-b53e-4389-8901-b84d79bb28c4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:16.698468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4fe22c39-b53e-4389-8901-b84d79bb28c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:16.912538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe22c39-b53e-4389-8901-b84d79bb28c4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:16.921350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c05f38-d51d-4ea9-9644-4a2aa56e65d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:16.922836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c05f38-d51d-4ea9-9644-4a2aa56e65d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:16.924593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c05f38-d51d-4ea9-9644-4a2aa56e65d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:16.925493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c05f38-d51d-4ea9-9644-4a2aa56e65d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:16.947328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c05f38-d51d-4ea9-9644-4a2aa56e65d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:16.951679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c05f38-d51d-4ea9-9644-4a2aa56e65d7 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:17.001099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 979/1971] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:52:17.371153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d4be1-e9e2-419f-a43a-985346b7be75 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:17.373513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-570d4be1-e9e2-419f-a43a-985346b7be75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:17.386574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c05f38-d51d-4ea9-9644-4a2aa56e65d7 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:17.390409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-60c05f38-d51d-4ea9-9644-4a2aa56e65d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:17.437584 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d4be1-e9e2-419f-a43a-985346b7be75 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:17.449048 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccb576-5cf7-44bb-89f0-86dde2383242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:17.450456 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccb576-5cf7-44bb-89f0-86dde2383242 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:17.451007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccb576-5cf7-44bb-89f0-86dde2383242 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:17.451544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccb576-5cf7-44bb-89f0-86dde2383242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:17.458139 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-12ccb576-5cf7-44bb-89f0-86dde2383242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:17.487048 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:17.619186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccb576-5cf7-44bb-89f0-86dde2383242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:17.622838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccb576-5cf7-44bb-89f0-86dde2383242 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:17.643808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c05f38-d51d-4ea9-9644-4a2aa56e65d7 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:17.658808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 993/1972] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:52:17.662643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:17.664016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:17.664527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:17.665020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] PATH_INFO: `/v3/projects/53b0d9fad85448439d2d8f4d1f0c17d3` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:17.667632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53b0d9fad85448439d2d8f4d1f0c17d3)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:17.671691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:17.849831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:17.864582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53b0d9fad85448439d2d8f4d1f0c17d3', 'request_id': 'req-7c307479-b371-424a-b232-59486e94df6f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:17.989672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53b0d9fad85448439d2d8f4d1f0c17d3', 'request_id': 'req-7c307479-b371-424a-b232-59486e94df6f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:18.018650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53b0d9fad85448439d2d8f4d1f0c17d3', 'request_id': 'req-7c307479-b371-424a-b232-59486e94df6f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:18.044507 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] The token cache is being invalidate because project 53b0d9fad85448439d2d8f4d1f0c17d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:18.045303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:18.047960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:18.052421 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 980/1973] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:17 2022] DELETE /identity/v3/projects/53b0d9fad85448439d2d8f4d1f0c17d3 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:18.053526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccb576-5cf7-44bb-89f0-86dde2383242 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:18.074452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccb576-5cf7-44bb-89f0-86dde2383242 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:18.196395 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240824a-3299-449a-927d-200992e0bb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:18.197213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240824a-3299-449a-927d-200992e0bb2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:18.197495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240824a-3299-449a-927d-200992e0bb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:18.197768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240824a-3299-449a-927d-200992e0bb2a admin admin] PATH_INFO: `/v3/users/e9ad1234c00046149c1774a0a0953a18` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:18.199084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240824a-3299-449a-927d-200992e0bb2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ad1234c00046149c1774a0a0953a18)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:18.201403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240824a-3299-449a-927d-200992e0bb2a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:18.459907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6240824a-3299-449a-927d-200992e0bb2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ad1234c00046149c1774a0a0953a18'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:18.497035 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6240824a-3299-449a-927d-200992e0bb2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ad1234c00046149c1774a0a0953a18'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:18.512601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6240824a-3299-449a-927d-200992e0bb2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ad1234c00046149c1774a0a0953a18'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:18.604674 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6240824a-3299-449a-927d-200992e0bb2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ad1234c00046149c1774a0a0953a18'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:18.607425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6240824a-3299-449a-927d-200992e0bb2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ad1234c00046149c1774a0a0953a18'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:18.634731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 994/1974] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:18 2022] DELETE /identity/v3/users/e9ad1234c00046149c1774a0a0953a18 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:19.587964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:19.590167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:19.606612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307479-b371-424a-b232-59486e94df6f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:19.730669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:19.731709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:19.732184 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:19.732652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] PATH_INFO: `/v3/projects/be2f012db59b4e1c9c1aad3d38ee0e23` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:19.736140 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2f012db59b4e1c9c1aad3d38ee0e23)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:19.740550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:19.916282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2f012db59b4e1c9c1aad3d38ee0e23', 'request_id': 'req-b8b6bc3c-797e-407d-a29b-9479cf8342bc'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:19.988911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2f012db59b4e1c9c1aad3d38ee0e23', 'request_id': 'req-b8b6bc3c-797e-407d-a29b-9479cf8342bc'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:19.998482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2f012db59b4e1c9c1aad3d38ee0e23', 'request_id': 'req-b8b6bc3c-797e-407d-a29b-9479cf8342bc'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:20.013865 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] The token cache is being invalidate because project be2f012db59b4e1c9c1aad3d38ee0e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:20.014236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:20.015607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b8b6bc3c-797e-407d-a29b-9479cf8342bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:20.018916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 981/1975] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:19 2022] DELETE /identity/v3/projects/be2f012db59b4e1c9c1aad3d38ee0e23 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:20.027392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dd62b-cb9f-45ca-8b67-c3a0d9c1bcaf None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:20.028321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dd62b-cb9f-45ca-8b67-c3a0d9c1bcaf None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:20.028321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dd62b-cb9f-45ca-8b67-c3a0d9c1bcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:20.065027 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b26dd62b-cb9f-45ca-8b67-c3a0d9c1bcaf None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:20.067243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b26dd62b-cb9f-45ca-8b67-c3a0d9c1bcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:20.119746 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:20.194635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 995/1976] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:20.204293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc1282-d31b-4f74-ace3-168cf26bc077 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:20.205556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc1282-d31b-4f74-ace3-168cf26bc077 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:20.206137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc1282-d31b-4f74-ace3-168cf26bc077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:20.244534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-14bc1282-d31b-4f74-ace3-168cf26bc077 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:20.246147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-14bc1282-d31b-4f74-ace3-168cf26bc077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:20.256327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 982/1977] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:20.262117 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dd62b-cb9f-45ca-8b67-c3a0d9c1bcaf None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:20.291145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dd62b-cb9f-45ca-8b67-c3a0d9c1bcaf None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:20.307077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ba671-a874-4963-a8a0-57fad7b54f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:20.307718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ba671-a874-4963-a8a0-57fad7b54f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:20.308214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ba671-a874-4963-a8a0-57fad7b54f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:20.308377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ba671-a874-4963-a8a0-57fad7b54f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:20.309864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ba671-a874-4963-a8a0-57fad7b54f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:52:20.309864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ba671-a874-4963-a8a0-57fad7b54f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:20.313060 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ba671-a874-4963-a8a0-57fad7b54f7b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:20.345013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 996/1978] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:52:20.352276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc1282-d31b-4f74-ace3-168cf26bc077 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:20.381399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc1282-d31b-4f74-ace3-168cf26bc077 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:20.395894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49a8ab-c68f-47d2-98a8-1981da4f4a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:20.397363 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49a8ab-c68f-47d2-98a8-1981da4f4a74 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:20.397816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49a8ab-c68f-47d2-98a8-1981da4f4a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:20.398480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49a8ab-c68f-47d2-98a8-1981da4f4a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:20.403433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49a8ab-c68f-47d2-98a8-1981da4f4a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:20.409112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49a8ab-c68f-47d2-98a8-1981da4f4a74 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:20.583682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 983/1979] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:20 2022] POST /identity/v3/projects => generated 365 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:20.584729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ba671-a874-4963-a8a0-57fad7b54f7b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:20.596859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ba671-a874-4963-a8a0-57fad7b54f7b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:20.703446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c95c67-05ab-42df-8908-eed764dd41a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:20.704326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c95c67-05ab-42df-8908-eed764dd41a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:20.704754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c95c67-05ab-42df-8908-eed764dd41a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:20.705348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c95c67-05ab-42df-8908-eed764dd41a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:20.706641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c95c67-05ab-42df-8908-eed764dd41a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:20.709805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c95c67-05ab-42df-8908-eed764dd41a1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:20.895809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 997/1980] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:20 2022] POST /identity/v3/users => generated 283 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:52:20.901133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49a8ab-c68f-47d2-98a8-1981da4f4a74 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:20.916627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49a8ab-c68f-47d2-98a8-1981da4f4a74 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:20.926200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6452-3e69-4c25-9511-39f0d186c6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:20.927252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6452-3e69-4c25-9511-39f0d186c6aa admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:20.928161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6452-3e69-4c25-9511-39f0d186c6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:20.928448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6452-3e69-4c25-9511-39f0d186c6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:20.929606 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a6452-3e69-4c25-9511-39f0d186c6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:20.932107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a6452-3e69-4c25-9511-39f0d186c6aa admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:20.952107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 984/1981] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:52:20.958324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c95c67-05ab-42df-8908-eed764dd41a1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:20.980900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c95c67-05ab-42df-8908-eed764dd41a1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:20.990540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d17f39-81d5-4f5e-8683-ebbcdc7f80b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:20.991530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d17f39-81d5-4f5e-8683-ebbcdc7f80b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:20.992033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d17f39-81d5-4f5e-8683-ebbcdc7f80b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:20.992521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d17f39-81d5-4f5e-8683-ebbcdc7f80b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:20.993756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d17f39-81d5-4f5e-8683-ebbcdc7f80b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:20.996810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d17f39-81d5-4f5e-8683-ebbcdc7f80b9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:21.013420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 998/1982] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:21.019207 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6452-3e69-4c25-9511-39f0d186c6aa admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:21.041982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6452-3e69-4c25-9511-39f0d186c6aa admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:21.051593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b728ce-672d-4615-8cd9-36a086cb3d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:21.052721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b728ce-672d-4615-8cd9-36a086cb3d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:21.053430 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b728ce-672d-4615-8cd9-36a086cb3d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:21.054207 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b728ce-672d-4615-8cd9-36a086cb3d43 admin admin] PATH_INFO: `/v3/projects/2214fedfecab49f09b957c3de06b91f7/users/c4bcd65ba21249c2a2f2b8af07191114/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:21.093059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b728ce-672d-4615-8cd9-36a086cb3d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2214fedfecab49f09b957c3de06b91f7, user_id=c4bcd65ba21249c2a2f2b8af07191114, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:21.097725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b728ce-672d-4615-8cd9-36a086cb3d43 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:21.104929 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:21.203503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 985/1983] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:21 2022] PUT /identity/v3/projects/2214fedfecab49f09b957c3de06b91f7/users/c4bcd65ba21249c2a2f2b8af07191114/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:21.214588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48279c-717f-4dfb-889a-786fd2d997f9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:21.218160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48279c-717f-4dfb-889a-786fd2d997f9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:21.218160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48279c-717f-4dfb-889a-786fd2d997f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:21.249559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6b48279c-717f-4dfb-889a-786fd2d997f9 None None] MFA Rules not processed for user `c4bcd65ba21249c2a2f2b8af07191114`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:21.251797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6b48279c-717f-4dfb-889a-786fd2d997f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:21.289141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b728ce-672d-4615-8cd9-36a086cb3d43 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:21.291451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-99b728ce-672d-4615-8cd9-36a086cb3d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:21.388842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 999/1984] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:21.394789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48279c-717f-4dfb-889a-786fd2d997f9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:21.438469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:21.465627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b728ce-672d-4615-8cd9-36a086cb3d43 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:21.476747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535ed2b-430f-4031-934f-8e6869d72a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:21.477990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ed2b-430f-4031-934f-8e6869d72a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:21.478783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ed2b-430f-4031-934f-8e6869d72a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:21.479276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ed2b-430f-4031-934f-8e6869d72a80 admin admin] PATH_INFO: `/v3/users/1c2ab9863f7141d4ab989822e9f1c133` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:21.481643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535ed2b-430f-4031-934f-8e6869d72a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c2ab9863f7141d4ab989822e9f1c133)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:21.486210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535ed2b-430f-4031-934f-8e6869d72a80 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:21.495651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48279c-717f-4dfb-889a-786fd2d997f9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:21.503914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c520cab-654e-4085-88ab-4d66c052b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:21.504663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c520cab-654e-4085-88ab-4d66c052b598 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:21.505027 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c520cab-654e-4085-88ab-4d66c052b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:21.505376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c520cab-654e-4085-88ab-4d66c052b598 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:21.506157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c520cab-654e-4085-88ab-4d66c052b598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:21.509091 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c520cab-654e-4085-88ab-4d66c052b598 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:21.666251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1000/1985] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:21 2022] POST /identity/v3/projects => generated 365 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:21.671294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c520cab-654e-4085-88ab-4d66c052b598 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:21.817976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c520cab-654e-4085-88ab-4d66c052b598 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:21.826363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f18ad-2df5-4243-8160-c0ae47777efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:21.827476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f18ad-2df5-4243-8160-c0ae47777efa admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:21.827950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f18ad-2df5-4243-8160-c0ae47777efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:21.828426 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f18ad-2df5-4243-8160-c0ae47777efa admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:21.830520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f18ad-2df5-4243-8160-c0ae47777efa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:21.836223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f18ad-2df5-4243-8160-c0ae47777efa admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:21.867884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1535ed2b-430f-4031-934f-8e6869d72a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c2ab9863f7141d4ab989822e9f1c133'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:21.883573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1535ed2b-430f-4031-934f-8e6869d72a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c2ab9863f7141d4ab989822e9f1c133'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:21.892886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1535ed2b-430f-4031-934f-8e6869d72a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c2ab9863f7141d4ab989822e9f1c133'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:21.963491 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1001/1986] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:21 2022] POST /identity/v3/users => generated 289 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:52:21.963491 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f18ad-2df5-4243-8160-c0ae47777efa admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:21.983152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f18ad-2df5-4243-8160-c0ae47777efa admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:21.993470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd4fa9-c114-4782-a78a-16b4d1487c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:21.994489 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd4fa9-c114-4782-a78a-16b4d1487c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:21.995021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd4fa9-c114-4782-a78a-16b4d1487c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:21.995473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd4fa9-c114-4782-a78a-16b4d1487c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:21.996636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd4fa9-c114-4782-a78a-16b4d1487c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:22.000266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd4fa9-c114-4782-a78a-16b4d1487c12 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:22.013699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1002/1987] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:52:22.020269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd4fa9-c114-4782-a78a-16b4d1487c12 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:22.033338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd4fa9-c114-4782-a78a-16b4d1487c12 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:22.039458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a03358-59ba-43b4-9d63-b6a2b9b89bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:22.040565 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a03358-59ba-43b4-9d63-b6a2b9b89bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:22.041327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a03358-59ba-43b4-9d63-b6a2b9b89bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:22.041721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1535ed2b-430f-4031-934f-8e6869d72a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c2ab9863f7141d4ab989822e9f1c133'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:22.042414 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a03358-59ba-43b4-9d63-b6a2b9b89bbe admin admin] PATH_INFO: `/v3/projects/7b5a3eb925a846b4be1c4d9082165c10/users/a42382836f254d2cafa2009a84d9b60d/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:22.044735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1535ed2b-430f-4031-934f-8e6869d72a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c2ab9863f7141d4ab989822e9f1c133'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:22.061443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 986/1988] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:21 2022] DELETE /identity/v3/users/1c2ab9863f7141d4ab989822e9f1c133 => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:22.071030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5196cf-776c-4eda-9e7b-cf4b7950a937 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:22.074535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5196cf-776c-4eda-9e7b-cf4b7950a937 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:22.075402 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5196cf-776c-4eda-9e7b-cf4b7950a937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:22.097483 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a03358-59ba-43b4-9d63-b6a2b9b89bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5a3eb925a846b4be1c4d9082165c10, user_id=a42382836f254d2cafa2009a84d9b60d, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:22.103496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a03358-59ba-43b4-9d63-b6a2b9b89bbe admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:22.118874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-dc5196cf-776c-4eda-9e7b-cf4b7950a937 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:22.120939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-dc5196cf-776c-4eda-9e7b-cf4b7950a937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:22.203642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 987/1989] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:22.214338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5196cf-776c-4eda-9e7b-cf4b7950a937 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:22.215940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-dc5196cf-776c-4eda-9e7b-cf4b7950a937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:22.238976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:22.275515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1003/1990] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:22 2022] PUT /identity/v3/projects/7b5a3eb925a846b4be1c4d9082165c10/users/a42382836f254d2cafa2009a84d9b60d/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:22.281803 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a03358-59ba-43b4-9d63-b6a2b9b89bbe admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:22.287647 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-63a03358-59ba-43b4-9d63-b6a2b9b89bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:22.328388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5196cf-776c-4eda-9e7b-cf4b7950a937 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:22.413628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a03358-59ba-43b4-9d63-b6a2b9b89bbe admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:22.419652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624a622-19e1-4303-afd7-d00b4f012ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:22.420322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624a622-19e1-4303-afd7-d00b4f012ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:22.420556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624a622-19e1-4303-afd7-d00b4f012ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:22.420827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624a622-19e1-4303-afd7-d00b4f012ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:22.426295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b12ad-2d04-4374-8e50-55526bec3588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:22.426943 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b12ad-2d04-4374-8e50-55526bec3588 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:22.427193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b12ad-2d04-4374-8e50-55526bec3588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:22.427568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b12ad-2d04-4374-8e50-55526bec3588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:22.428112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b12ad-2d04-4374-8e50-55526bec3588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:22.430411 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b12ad-2d04-4374-8e50-55526bec3588 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:22.439849 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:22.442118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624a622-19e1-4303-afd7-d00b4f012ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:22.443685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624a622-19e1-4303-afd7-d00b4f012ea3 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:22.450269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1004/1991] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:22.455730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b12ad-2d04-4374-8e50-55526bec3588 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:22.459804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 988/1992] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:52:22.474045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b12ad-2d04-4374-8e50-55526bec3588 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:22.477562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd122cbc-8e0f-47fc-b9ba-2cc3a56cc046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:22.478535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd122cbc-8e0f-47fc-b9ba-2cc3a56cc046 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:22.478961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd122cbc-8e0f-47fc-b9ba-2cc3a56cc046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:22.479492 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd122cbc-8e0f-47fc-b9ba-2cc3a56cc046 admin admin] PATH_INFO: `/v3/projects/7b5a3eb925a846b4be1c4d9082165c10/users/a42382836f254d2cafa2009a84d9b60d/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:22.481838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd122cbc-8e0f-47fc-b9ba-2cc3a56cc046 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5a3eb925a846b4be1c4d9082165c10, user_id=a42382836f254d2cafa2009a84d9b60d, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:22.485049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd122cbc-8e0f-47fc-b9ba-2cc3a56cc046 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:22.498935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1005/1993] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:22 2022] PUT /identity/v3/projects/7b5a3eb925a846b4be1c4d9082165c10/users/a42382836f254d2cafa2009a84d9b60d/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:22.510077 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cc57e-2c51-4fa6-80d5-6d2a190ada71 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:22.510681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cc57e-2c51-4fa6-80d5-6d2a190ada71 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:22.511040 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cc57e-2c51-4fa6-80d5-6d2a190ada71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:22.558597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-264cc57e-2c51-4fa6-80d5-6d2a190ada71 None None] MFA Rules not processed for user `a42382836f254d2cafa2009a84d9b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:22.558597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-264cc57e-2c51-4fa6-80d5-6d2a190ada71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:22.683967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 989/1994] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:22.703941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd122cbc-8e0f-47fc-b9ba-2cc3a56cc046 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:22.706982 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cd122cbc-8e0f-47fc-b9ba-2cc3a56cc046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:22.834838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd122cbc-8e0f-47fc-b9ba-2cc3a56cc046 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:22.844178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7137c-a75a-44a6-ab03-e363f75b8842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:22.844848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7137c-a75a-44a6-ab03-e363f75b8842 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:22.845673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7137c-a75a-44a6-ab03-e363f75b8842 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:22.846132 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7137c-a75a-44a6-ab03-e363f75b8842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:22.865209 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7137c-a75a-44a6-ab03-e363f75b8842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:22.868082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7137c-a75a-44a6-ab03-e363f75b8842 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:22.881680 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1006/1995] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:23.132178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cc57e-2c51-4fa6-80d5-6d2a190ada71 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:23.137224 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-264cc57e-2c51-4fa6-80d5-6d2a190ada71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:23.166965 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7137c-a75a-44a6-ab03-e363f75b8842 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:23.195681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7137c-a75a-44a6-ab03-e363f75b8842 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:23.208149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3cf02a-71dd-428f-a88d-a16826443c7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:23.210139 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cf02a-71dd-428f-a88d-a16826443c7c service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:23.210594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cf02a-71dd-428f-a88d-a16826443c7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:23.211038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cf02a-71dd-428f-a88d-a16826443c7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:23.215837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4c3cf02a-71dd-428f-a88d-a16826443c7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:23.287581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cc57e-2c51-4fa6-80d5-6d2a190ada71 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:23.299473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:23.305315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:23.305315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:23.305315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] PATH_INFO: `/v3/projects/7e8059926afb458b96a7af4b2acacdf1` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:23.305315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e8059926afb458b96a7af4b2acacdf1)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:23.305994 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:23.308005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:23.351274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cf02a-71dd-428f-a88d-a16826443c7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:23.360117 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cf02a-71dd-428f-a88d-a16826443c7c service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:23.374978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1007/1996] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:23.487468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e8059926afb458b96a7af4b2acacdf1', 'request_id': 'req-d45c5928-8ae1-4745-b6c4-3824174a113c'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:23.582705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e8059926afb458b96a7af4b2acacdf1', 'request_id': 'req-d45c5928-8ae1-4745-b6c4-3824174a113c'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:23.593135 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e8059926afb458b96a7af4b2acacdf1', 'request_id': 'req-d45c5928-8ae1-4745-b6c4-3824174a113c'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:23.611060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] The token cache is being invalidate because project 7e8059926afb458b96a7af4b2acacdf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:23.611360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:23.612027 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:23.613640 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 990/1997] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:23 2022] DELETE /identity/v3/projects/7e8059926afb458b96a7af4b2acacdf1 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:23.618777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3cf02a-71dd-428f-a88d-a16826443c7c service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:23.633450 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3cf02a-71dd-428f-a88d-a16826443c7c service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:23.730903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ade365-e067-4356-8ac6-9b62f250e654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:23.731781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ade365-e067-4356-8ac6-9b62f250e654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:23.732246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ade365-e067-4356-8ac6-9b62f250e654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:23.732702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ade365-e067-4356-8ac6-9b62f250e654 admin admin] PATH_INFO: `/v3/users/39d85a67f679436c9015067427e3542e` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:23.734202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ade365-e067-4356-8ac6-9b62f250e654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d85a67f679436c9015067427e3542e)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:23.737355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ade365-e067-4356-8ac6-9b62f250e654 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:24.024214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:24.044165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-69ade365-e067-4356-8ac6-9b62f250e654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d85a67f679436c9015067427e3542e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:24.063658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-69ade365-e067-4356-8ac6-9b62f250e654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d85a67f679436c9015067427e3542e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:24.074404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-69ade365-e067-4356-8ac6-9b62f250e654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d85a67f679436c9015067427e3542e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:24.177931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-69ade365-e067-4356-8ac6-9b62f250e654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d85a67f679436c9015067427e3542e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:24.178984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-69ade365-e067-4356-8ac6-9b62f250e654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d85a67f679436c9015067427e3542e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:24.197275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1008/1998] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:23 2022] DELETE /identity/v3/users/39d85a67f679436c9015067427e3542e => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:25.071711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:25.074116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:25.104346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c5928-8ae1-4745-b6c4-3824174a113c admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:25.230459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:25.232037 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:25.232608 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:25.233167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] PATH_INFO: `/v3/projects/1cf62861d9ad43a08f4c1ccb576e6d1b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:25.235421 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf62861d9ad43a08f4c1ccb576e6d1b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:25.244414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:25.462929 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf62861d9ad43a08f4c1ccb576e6d1b', 'request_id': 'req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:25.577888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf62861d9ad43a08f4c1ccb576e6d1b', 'request_id': 'req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:25.593440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf62861d9ad43a08f4c1ccb576e6d1b', 'request_id': 'req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:25.615993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] The token cache is being invalidate because project 1cf62861d9ad43a08f4c1ccb576e6d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:25.616453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:25.617263 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5fa7dbd7-e0d9-4360-905c-344e1878ddb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:25.619328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 991/1999] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:25 2022] DELETE /identity/v3/projects/1cf62861d9ad43a08f4c1ccb576e6d1b => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:25.625482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6ad3e-425d-4271-9a66-11f933687eea None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:25.627105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6ad3e-425d-4271-9a66-11f933687eea None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:25.627676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6ad3e-425d-4271-9a66-11f933687eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:25.665964 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0de6ad3e-425d-4271-9a66-11f933687eea None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:25.668655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0de6ad3e-425d-4271-9a66-11f933687eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:25.825906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1009/2000] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:25.834371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e337a6-b53d-4529-afe6-161bdf1f242c None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:25.838280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e337a6-b53d-4529-afe6-161bdf1f242c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:25.838280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e337a6-b53d-4529-afe6-161bdf1f242c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:25.863599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f2e337a6-b53d-4529-afe6-161bdf1f242c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:25.865819 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f2e337a6-b53d-4529-afe6-161bdf1f242c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:25.881246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 992/2001] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:52:25.887730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6ad3e-425d-4271-9a66-11f933687eea None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:25.898969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6ad3e-425d-4271-9a66-11f933687eea None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:25.904776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fb019-c9a5-4d8f-b162-2569375a05cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:25.905852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fb019-c9a5-4d8f-b162-2569375a05cb admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:25.906522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fb019-c9a5-4d8f-b162-2569375a05cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:25.907547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fb019-c9a5-4d8f-b162-2569375a05cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:25.908744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fb019-c9a5-4d8f-b162-2569375a05cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:52:25.909280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fb019-c9a5-4d8f-b162-2569375a05cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:25.913058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fb019-c9a5-4d8f-b162-2569375a05cb admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:25.936589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1010/2002] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:52:25.942599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e337a6-b53d-4529-afe6-161bdf1f242c None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:25.959374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e337a6-b53d-4529-afe6-161bdf1f242c None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:25.964618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b15bef-b9f5-49c9-a6b8-dd0df662b9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:25.965409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b15bef-b9f5-49c9-a6b8-dd0df662b9de admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:25.965695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b15bef-b9f5-49c9-a6b8-dd0df662b9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:25.965970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b15bef-b9f5-49c9-a6b8-dd0df662b9de admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:25.966662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b15bef-b9f5-49c9-a6b8-dd0df662b9de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:25.968631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b15bef-b9f5-49c9-a6b8-dd0df662b9de admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:25.983418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:26.134338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 993/2003] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:25 2022] POST /identity/v3/projects => generated 375 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:26.140842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fb019-c9a5-4d8f-b162-2569375a05cb admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:26.164355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fb019-c9a5-4d8f-b162-2569375a05cb admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:26.244465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666447ce-c976-4572-ad5b-210e0a360656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:26.245208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666447ce-c976-4572-ad5b-210e0a360656 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:26.245531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666447ce-c976-4572-ad5b-210e0a360656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:26.245847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666447ce-c976-4572-ad5b-210e0a360656 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:26.246727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666447ce-c976-4572-ad5b-210e0a360656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:26.248767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666447ce-c976-4572-ad5b-210e0a360656 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:26.385420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1011/2004] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:26 2022] POST /identity/v3/users => generated 288 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:52:26.386986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b15bef-b9f5-49c9-a6b8-dd0df662b9de admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:26.404375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b15bef-b9f5-49c9-a6b8-dd0df662b9de admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:26.412870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810a81f-4639-45a2-a073-b0d4f54fec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:26.413858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810a81f-4639-45a2-a073-b0d4f54fec6f admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:26.414318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810a81f-4639-45a2-a073-b0d4f54fec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:26.414870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810a81f-4639-45a2-a073-b0d4f54fec6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:26.416040 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810a81f-4639-45a2-a073-b0d4f54fec6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:26.419013 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810a81f-4639-45a2-a073-b0d4f54fec6f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:26.436027 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 994/2005] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:26.441881 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666447ce-c976-4572-ad5b-210e0a360656 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:26.458303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666447ce-c976-4572-ad5b-210e0a360656 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:26.466624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c2685-3114-4c0d-865c-fa125e55c3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:26.467388 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2685-3114-4c0d-865c-fa125e55c3db admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:26.467758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2685-3114-4c0d-865c-fa125e55c3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:26.468250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2685-3114-4c0d-865c-fa125e55c3db admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:26.469453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c2685-3114-4c0d-865c-fa125e55c3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:26.473372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c2685-3114-4c0d-865c-fa125e55c3db admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:26.499517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1012/2006] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:52:26.502741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810a81f-4639-45a2-a073-b0d4f54fec6f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:26.518325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810a81f-4639-45a2-a073-b0d4f54fec6f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:26.525834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40023575-ab16-413b-8f00-5e88e538fd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:26.526967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40023575-ab16-413b-8f00-5e88e538fd8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:26.527455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40023575-ab16-413b-8f00-5e88e538fd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:26.527927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40023575-ab16-413b-8f00-5e88e538fd8a admin admin] PATH_INFO: `/v3/projects/5c55153ffeb043369b065d97a293fbde/users/3fe48ba43441416ba087670aac68a3ab/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:26.547472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40023575-ab16-413b-8f00-5e88e538fd8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c55153ffeb043369b065d97a293fbde, user_id=3fe48ba43441416ba087670aac68a3ab, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:26.551102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40023575-ab16-413b-8f00-5e88e538fd8a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:26.625006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 995/2007] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:26 2022] PUT /identity/v3/projects/5c55153ffeb043369b065d97a293fbde/users/3fe48ba43441416ba087670aac68a3ab/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:26.640163 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b6646-b69e-433d-be29-9f6fb281e8a3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:26.640789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b6646-b69e-433d-be29-9f6fb281e8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:26.640789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b6646-b69e-433d-be29-9f6fb281e8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:26.670749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:26.675499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-892b6646-b69e-433d-be29-9f6fb281e8a3 None None] MFA Rules not processed for user `3fe48ba43441416ba087670aac68a3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:26.676813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-892b6646-b69e-433d-be29-9f6fb281e8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:26.764031 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1013/2008] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:26.769830 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40023575-ab16-413b-8f00-5e88e538fd8a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:26.880336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40023575-ab16-413b-8f00-5e88e538fd8a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:26.888653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152ed4c-df7c-4b68-803e-3eec9ac764a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:26.889736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152ed4c-df7c-4b68-803e-3eec9ac764a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:26.890200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152ed4c-df7c-4b68-803e-3eec9ac764a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:26.890706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152ed4c-df7c-4b68-803e-3eec9ac764a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:26.892580 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152ed4c-df7c-4b68-803e-3eec9ac764a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:26.896388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152ed4c-df7c-4b68-803e-3eec9ac764a6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:26.991127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 996/2009] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:26 2022] POST /identity/v3/projects => generated 377 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:26.995580 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b6646-b69e-433d-be29-9f6fb281e8a3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:27.007368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b6646-b69e-433d-be29-9f6fb281e8a3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:27.077714 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe79153-d39c-4f0b-b083-93f6a7d92d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:27.078568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe79153-d39c-4f0b-b083-93f6a7d92d1d admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:27.079547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe79153-d39c-4f0b-b083-93f6a7d92d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:27.079989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe79153-d39c-4f0b-b083-93f6a7d92d1d admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:27.080566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe79153-d39c-4f0b-b083-93f6a7d92d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:27.082653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe79153-d39c-4f0b-b083-93f6a7d92d1d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:27.171868 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1014/2010] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:26 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:52:27.179672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152ed4c-df7c-4b68-803e-3eec9ac764a6 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:27.198059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152ed4c-df7c-4b68-803e-3eec9ac764a6 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:27.208189 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d90231-e1ba-4de2-b7bb-13b0b8f6712b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:27.209123 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d90231-e1ba-4de2-b7bb-13b0b8f6712b admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:27.209762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d90231-e1ba-4de2-b7bb-13b0b8f6712b admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:27.210177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d90231-e1ba-4de2-b7bb-13b0b8f6712b admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:27.211328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d90231-e1ba-4de2-b7bb-13b0b8f6712b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:27.214219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d90231-e1ba-4de2-b7bb-13b0b8f6712b admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:27.244185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 997/2011] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:27.252333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe79153-d39c-4f0b-b083-93f6a7d92d1d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:27.269145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe79153-d39c-4f0b-b083-93f6a7d92d1d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:27.286662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dffc9e-26c7-4d4c-8a26-939c6672d7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:27.287467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dffc9e-26c7-4d4c-8a26-939c6672d7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:27.287846 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dffc9e-26c7-4d4c-8a26-939c6672d7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:27.288218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dffc9e-26c7-4d4c-8a26-939c6672d7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:27.288986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dffc9e-26c7-4d4c-8a26-939c6672d7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:27.292237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dffc9e-26c7-4d4c-8a26-939c6672d7f7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:27.312912 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1015/2012] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:27.321549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d90231-e1ba-4de2-b7bb-13b0b8f6712b admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:27.337263 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d90231-e1ba-4de2-b7bb-13b0b8f6712b admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:27.342376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af128db0-15e9-4c4b-a842-2de92fa8514e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:27.343098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af128db0-15e9-4c4b-a842-2de92fa8514e admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:27.343376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af128db0-15e9-4c4b-a842-2de92fa8514e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:27.344101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af128db0-15e9-4c4b-a842-2de92fa8514e admin admin] PATH_INFO: `/v3/projects/71488192c3bf44d69a45994d247a5665/users/1390e1179a9e48fea9f34f6b685f0e1e/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:27.371279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af128db0-15e9-4c4b-a842-2de92fa8514e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71488192c3bf44d69a45994d247a5665, user_id=1390e1179a9e48fea9f34f6b685f0e1e, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:27.375079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af128db0-15e9-4c4b-a842-2de92fa8514e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:27.483430 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 998/2013] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:27 2022] PUT /identity/v3/projects/71488192c3bf44d69a45994d247a5665/users/1390e1179a9e48fea9f34f6b685f0e1e/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:27.504302 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386ae7a-76ed-4ae7-870f-481ac4f47a14 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:27.504715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386ae7a-76ed-4ae7-870f-481ac4f47a14 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:27.505079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386ae7a-76ed-4ae7-870f-481ac4f47a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:27.546325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a386ae7a-76ed-4ae7-870f-481ac4f47a14 None None] MFA Rules not processed for user `1390e1179a9e48fea9f34f6b685f0e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:27.549059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a386ae7a-76ed-4ae7-870f-481ac4f47a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:27.650559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1016/2014] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:27.658404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af128db0-15e9-4c4b-a842-2de92fa8514e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:27.770837 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af128db0-15e9-4c4b-a842-2de92fa8514e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:27.776106 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b8071-97e0-452d-8151-9fd72a6b82a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:27.776704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b8071-97e0-452d-8151-9fd72a6b82a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:27.777033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b8071-97e0-452d-8151-9fd72a6b82a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:27.777514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b8071-97e0-452d-8151-9fd72a6b82a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:27.778252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b8071-97e0-452d-8151-9fd72a6b82a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:27.780408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b8071-97e0-452d-8151-9fd72a6b82a8 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:27.794242 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:27.931412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 999/2015] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:27 2022] POST /identity/v3/projects => generated 379 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:52:27.932778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386ae7a-76ed-4ae7-870f-481ac4f47a14 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:27.946538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386ae7a-76ed-4ae7-870f-481ac4f47a14 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:27.959739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:28.022177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b8348-e765-404f-85cc-0e4265ad13c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:28.022958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8348-e765-404f-85cc-0e4265ad13c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:28.023285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8348-e765-404f-85cc-0e4265ad13c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:28.023606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8348-e765-404f-85cc-0e4265ad13c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:28.024337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b8348-e765-404f-85cc-0e4265ad13c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:28.027055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b8348-e765-404f-85cc-0e4265ad13c9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:28.143835 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1017/2016] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:27 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:28.144726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b8071-97e0-452d-8151-9fd72a6b82a8 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:28.167091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b8071-97e0-452d-8151-9fd72a6b82a8 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:28.175977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc7d8b-2209-4aa8-92f9-ab5e4f3d19b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:28.177177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc7d8b-2209-4aa8-92f9-ab5e4f3d19b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:28.178897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc7d8b-2209-4aa8-92f9-ab5e4f3d19b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:28.179430 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc7d8b-2209-4aa8-92f9-ab5e4f3d19b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:28.180317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc7d8b-2209-4aa8-92f9-ab5e4f3d19b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:28.182913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc7d8b-2209-4aa8-92f9-ab5e4f3d19b2 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:28.199975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1000/2017] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:28.206749 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b8348-e765-404f-85cc-0e4265ad13c9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:28.220760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b8348-e765-404f-85cc-0e4265ad13c9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:28.227557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02539d12-3e41-4a7d-8550-12d376cf8962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:28.228247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02539d12-3e41-4a7d-8550-12d376cf8962 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:28.228526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02539d12-3e41-4a7d-8550-12d376cf8962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:28.228815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02539d12-3e41-4a7d-8550-12d376cf8962 admin admin] PATH_INFO: `/v3/projects/e55ea4203d624c3ea06854ec6adf92be/users/3de599844987433f9dd4cc98aadf9d61/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:28.248997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02539d12-3e41-4a7d-8550-12d376cf8962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55ea4203d624c3ea06854ec6adf92be, user_id=3de599844987433f9dd4cc98aadf9d61, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:28.252182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02539d12-3e41-4a7d-8550-12d376cf8962 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:28.427044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1018/2018] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:28 2022] PUT /identity/v3/projects/e55ea4203d624c3ea06854ec6adf92be/users/3de599844987433f9dd4cc98aadf9d61/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:28.431298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc7d8b-2209-4aa8-92f9-ab5e4f3d19b2 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:28.448657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc7d8b-2209-4aa8-92f9-ab5e4f3d19b2 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:28.564953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12777186-e7b6-450e-aee5-7abb816367e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:28.565592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12777186-e7b6-450e-aee5-7abb816367e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:28.566005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12777186-e7b6-450e-aee5-7abb816367e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:28.566312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12777186-e7b6-450e-aee5-7abb816367e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:28.567171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12777186-e7b6-450e-aee5-7abb816367e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:28.568958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12777186-e7b6-450e-aee5-7abb816367e4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:28.580991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1001/2019] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:52:28.586474 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02539d12-3e41-4a7d-8550-12d376cf8962 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:28.612526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02539d12-3e41-4a7d-8550-12d376cf8962 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:28.618395 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089caf8-91f9-4850-8d58-a549b8534a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:28.619134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089caf8-91f9-4850-8d58-a549b8534a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:28.619756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089caf8-91f9-4850-8d58-a549b8534a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:28.619756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089caf8-91f9-4850-8d58-a549b8534a5b admin admin] PATH_INFO: `/v3/projects/e55ea4203d624c3ea06854ec6adf92be/users/3de599844987433f9dd4cc98aadf9d61/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:28.621894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089caf8-91f9-4850-8d58-a549b8534a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55ea4203d624c3ea06854ec6adf92be, user_id=3de599844987433f9dd4cc98aadf9d61, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:28.624772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089caf8-91f9-4850-8d58-a549b8534a5b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:28.644256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1019/2020] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:28 2022] PUT /identity/v3/projects/e55ea4203d624c3ea06854ec6adf92be/users/3de599844987433f9dd4cc98aadf9d61/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:28.662667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5989e-c265-422b-a221-9feb2bec88a2 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:28.663015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5989e-c265-422b-a221-9feb2bec88a2 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:28.663289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5989e-c265-422b-a221-9feb2bec88a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:28.717938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-afd5989e-c265-422b-a221-9feb2bec88a2 None None] MFA Rules not processed for user `3de599844987433f9dd4cc98aadf9d61`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:28.721360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-afd5989e-c265-422b-a221-9feb2bec88a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:28.851281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1002/2021] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:52:28.916103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089caf8-91f9-4850-8d58-a549b8534a5b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:28.919254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8089caf8-91f9-4850-8d58-a549b8534a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:29.052910 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089caf8-91f9-4850-8d58-a549b8534a5b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:29.058620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7628b-74e9-4084-b30e-155c61f9693c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:29.059452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7628b-74e9-4084-b30e-155c61f9693c service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:29.059965 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7628b-74e9-4084-b30e-155c61f9693c service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:29.060446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7628b-74e9-4084-b30e-155c61f9693c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:29.075231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7628b-74e9-4084-b30e-155c61f9693c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:29.077574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7628b-74e9-4084-b30e-155c61f9693c service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:29.091191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1020/2022] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:29.254820 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5989e-c265-422b-a221-9feb2bec88a2 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:29.266788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5989e-c265-422b-a221-9feb2bec88a2 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:29.284038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6aeba-bf45-4991-b9eb-767d33c31683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:29.285534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6aeba-bf45-4991-b9eb-767d33c31683 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:29.286557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6aeba-bf45-4991-b9eb-767d33c31683 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:29.287245 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6aeba-bf45-4991-b9eb-767d33c31683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:29.389120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6aeba-bf45-4991-b9eb-767d33c31683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:29.390630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6aeba-bf45-4991-b9eb-767d33c31683 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:29.401212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:29.403900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1003/2023] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:32.543961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7628b-74e9-4084-b30e-155c61f9693c service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:32.547279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-64d7628b-74e9-4084-b30e-155c61f9693c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:32.644296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7628b-74e9-4084-b30e-155c61f9693c service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:32.655996 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:32.658857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:32.659502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:32.659933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] PATH_INFO: `/v3/users/c4bcd65ba21249c2a2f2b8af07191114` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:32.665194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4bcd65ba21249c2a2f2b8af07191114)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:32.669347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:32.680661 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:32.956008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4bcd65ba21249c2a2f2b8af07191114'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:32.990151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4bcd65ba21249c2a2f2b8af07191114'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:33.000340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4bcd65ba21249c2a2f2b8af07191114'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:33.113452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4bcd65ba21249c2a2f2b8af07191114'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:33.114308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4bcd65ba21249c2a2f2b8af07191114'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:33.124872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1021/2024] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:32 2022] DELETE /identity/v3/users/c4bcd65ba21249c2a2f2b8af07191114 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:33.132303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672a47-3371-4517-8f7e-d1ee11dbcc6f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:33.133230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672a47-3371-4517-8f7e-d1ee11dbcc6f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:33.133396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672a47-3371-4517-8f7e-d1ee11dbcc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:33.177508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-c6672a47-3371-4517-8f7e-d1ee11dbcc6f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:33.179903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c6672a47-3371-4517-8f7e-d1ee11dbcc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:33.276556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1004/2025] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:33.290697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:33.292669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:33.402618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fe0a7-44b7-4084-bc04-08070d733843 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:33.412937 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999af88a-eaf6-42b1-9efb-a969cda1ede8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:33.414012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999af88a-eaf6-42b1-9efb-a969cda1ede8 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:33.414464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999af88a-eaf6-42b1-9efb-a969cda1ede8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:33.416387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999af88a-eaf6-42b1-9efb-a969cda1ede8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:33.428497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999af88a-eaf6-42b1-9efb-a969cda1ede8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:33.430094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999af88a-eaf6-42b1-9efb-a969cda1ede8 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:33.451025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1022/2026] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:52:34.257761 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6672a47-3371-4517-8f7e-d1ee11dbcc6f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:34.260513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c6672a47-3371-4517-8f7e-d1ee11dbcc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:34.279016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6672a47-3371-4517-8f7e-d1ee11dbcc6f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:34.287609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:34.288625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:34.288896 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:34.289417 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] PATH_INFO: `/v3/projects/2214fedfecab49f09b957c3de06b91f7` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:34.291784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2214fedfecab49f09b957c3de06b91f7)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:34.293752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:34.447895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2214fedfecab49f09b957c3de06b91f7', 'request_id': 'req-a306abe7-4bca-4edb-8a96-15f0826a7065'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:34.498633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999af88a-eaf6-42b1-9efb-a969cda1ede8 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:34.500521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-999af88a-eaf6-42b1-9efb-a969cda1ede8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:34.514239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999af88a-eaf6-42b1-9efb-a969cda1ede8 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:34.553612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2214fedfecab49f09b957c3de06b91f7', 'request_id': 'req-a306abe7-4bca-4edb-8a96-15f0826a7065'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:34.565550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2214fedfecab49f09b957c3de06b91f7', 'request_id': 'req-a306abe7-4bca-4edb-8a96-15f0826a7065'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:34.591759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] The token cache is being invalidate because project 2214fedfecab49f09b957c3de06b91f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:34.592280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:34.593233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f90180-e2b2-4688-9e37-e43f4457ed14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:34.593425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:34.594399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f90180-e2b2-4688-9e37-e43f4457ed14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:34.594965 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f90180-e2b2-4688-9e37-e43f4457ed14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:34.595372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f90180-e2b2-4688-9e37-e43f4457ed14 admin admin] PATH_INFO: `/v3/users/3fe48ba43441416ba087670aac68a3ab` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:34.596393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1005/2027] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:34 2022] DELETE /identity/v3/projects/2214fedfecab49f09b957c3de06b91f7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:34.596759 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f90180-e2b2-4688-9e37-e43f4457ed14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fe48ba43441416ba087670aac68a3ab)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:34.599376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f90180-e2b2-4688-9e37-e43f4457ed14 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:34.604647 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:34.610132 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:34.638237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306abe7-4bca-4edb-8a96-15f0826a7065 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:34.649160 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:34.650520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:34.651007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:34.651544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] PATH_INFO: `/v3/users/a42382836f254d2cafa2009a84d9b60d` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:34.653867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a42382836f254d2cafa2009a84d9b60d)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:34.658727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:34.858336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:34.878747 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-58f90180-e2b2-4688-9e37-e43f4457ed14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fe48ba43441416ba087670aac68a3ab'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:34.913344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-58f90180-e2b2-4688-9e37-e43f4457ed14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fe48ba43441416ba087670aac68a3ab'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:34.936043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-58f90180-e2b2-4688-9e37-e43f4457ed14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fe48ba43441416ba087670aac68a3ab'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:34.961266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a42382836f254d2cafa2009a84d9b60d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:34.986271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a42382836f254d2cafa2009a84d9b60d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:34.999455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a42382836f254d2cafa2009a84d9b60d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:35.006557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:35.068974 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-58f90180-e2b2-4688-9e37-e43f4457ed14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fe48ba43441416ba087670aac68a3ab'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:35.070123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-58f90180-e2b2-4688-9e37-e43f4457ed14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fe48ba43441416ba087670aac68a3ab'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:35.089453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1023/2028] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:34 2022] DELETE /identity/v3/users/3fe48ba43441416ba087670aac68a3ab => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:35.094187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a7f2b-a926-4c82-b0c6-a67888f9ba34 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:35.094867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a7f2b-a926-4c82-b0c6-a67888f9ba34 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:35.095469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a7f2b-a926-4c82-b0c6-a67888f9ba34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:35.160857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a42382836f254d2cafa2009a84d9b60d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:35.162002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a42382836f254d2cafa2009a84d9b60d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:35.188944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1006/2029] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:34 2022] DELETE /identity/v3/users/a42382836f254d2cafa2009a84d9b60d => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:35.213917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e64a7f2b-a926-4c82-b0c6-a67888f9ba34 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:35.220722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e64a7f2b-a926-4c82-b0c6-a67888f9ba34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:35.446652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1024/2030] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:35.465110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:35.467397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:35.542217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f18811-ced2-4b2d-99a9-9f2f3f93a840 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:35.550057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88daac-5084-413b-bbfb-69d867a4f020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:35.550751 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88daac-5084-413b-bbfb-69d867a4f020 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:35.551046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88daac-5084-413b-bbfb-69d867a4f020 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:35.551324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88daac-5084-413b-bbfb-69d867a4f020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:35.575198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88daac-5084-413b-bbfb-69d867a4f020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:35.578299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88daac-5084-413b-bbfb-69d867a4f020 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:35.598669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1007/2031] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:36.505884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a7f2b-a926-4c82-b0c6-a67888f9ba34 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:36.508089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e64a7f2b-a926-4c82-b0c6-a67888f9ba34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:36.526064 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a7f2b-a926-4c82-b0c6-a67888f9ba34 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:36.532882 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:36.534186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:36.534833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:36.535449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] PATH_INFO: `/v3/projects/5c55153ffeb043369b065d97a293fbde` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:36.537146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c55153ffeb043369b065d97a293fbde)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:36.540063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:36.768088 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88daac-5084-413b-bbfb-69d867a4f020 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:36.770898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-df88daac-5084-413b-bbfb-69d867a4f020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:36.772345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c55153ffeb043369b065d97a293fbde', 'request_id': 'req-d81057b3-a899-411a-8a5c-f1c0b23617bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:36.800819 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c55153ffeb043369b065d97a293fbde', 'request_id': 'req-d81057b3-a899-411a-8a5c-f1c0b23617bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:36.837532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88daac-5084-413b-bbfb-69d867a4f020 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:36.896826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c55153ffeb043369b065d97a293fbde', 'request_id': 'req-d81057b3-a899-411a-8a5c-f1c0b23617bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:36.931352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] The token cache is being invalidate because project 5c55153ffeb043369b065d97a293fbde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:36.931869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:36.933328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:36.946287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1025/2032] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:36 2022] DELETE /identity/v3/projects/5c55153ffeb043369b065d97a293fbde => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:36.949634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:36.954967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:37.064131 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:37.067595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:37.070551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:37.071548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:37.072299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] PATH_INFO: `/v3/projects/7b5a3eb925a846b4be1c4d9082165c10` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:37.079853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b5a3eb925a846b4be1c4d9082165c10)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:37.084994 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:37.188238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81057b3-a899-411a-8a5c-f1c0b23617bd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:37.197762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:37.199000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:37.200736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:37.201264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] PATH_INFO: `/v3/users/1390e1179a9e48fea9f34f6b685f0e1e` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:37.204479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1390e1179a9e48fea9f34f6b685f0e1e)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:37.209015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:37.322939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b5a3eb925a846b4be1c4d9082165c10', 'request_id': 'req-3c538502-8ab5-4716-94eb-b3efbcaa6664'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:37.461828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b5a3eb925a846b4be1c4d9082165c10', 'request_id': 'req-3c538502-8ab5-4716-94eb-b3efbcaa6664'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:37.471555 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b5a3eb925a846b4be1c4d9082165c10', 'request_id': 'req-3c538502-8ab5-4716-94eb-b3efbcaa6664'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:37.498526 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] The token cache is being invalidate because project 7b5a3eb925a846b4be1c4d9082165c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:37.499319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:37.501854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3c538502-8ab5-4716-94eb-b3efbcaa6664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:37.504199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1008/2033] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:36 2022] DELETE /identity/v3/projects/7b5a3eb925a846b4be1c4d9082165c10 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:37.514465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6732b7-d8bc-4a63-8a84-ef3759f4e147 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:37.514465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6732b7-d8bc-4a63-8a84-ef3759f4e147 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:37.514923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6732b7-d8bc-4a63-8a84-ef3759f4e147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:37.553456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-6b6732b7-d8bc-4a63-8a84-ef3759f4e147 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:37.559558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6b6732b7-d8bc-4a63-8a84-ef3759f4e147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:37.567855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1390e1179a9e48fea9f34f6b685f0e1e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:37.608308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1390e1179a9e48fea9f34f6b685f0e1e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:37.624300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:37.630815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1390e1179a9e48fea9f34f6b685f0e1e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:37.684547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1009/2034] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:37.698830 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250b8c6-368c-49d3-b216-fe30bd4197d0 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:37.699320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250b8c6-368c-49d3-b216-fe30bd4197d0 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:37.699804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250b8c6-368c-49d3-b216-fe30bd4197d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:37.754444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-5250b8c6-368c-49d3-b216-fe30bd4197d0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:37.756810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5250b8c6-368c-49d3-b216-fe30bd4197d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:37.780681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1010/2035] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:52:37.789220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250b8c6-368c-49d3-b216-fe30bd4197d0 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:37.796285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1390e1179a9e48fea9f34f6b685f0e1e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:37.796285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1390e1179a9e48fea9f34f6b685f0e1e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:37.804371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1026/2036] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:36 2022] DELETE /identity/v3/users/1390e1179a9e48fea9f34f6b685f0e1e => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:37.822316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250b8c6-368c-49d3-b216-fe30bd4197d0 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:37.961197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2546d-a20c-410d-b1ba-9f05b1db392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:37.962316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2546d-a20c-410d-b1ba-9f05b1db392d admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:37.963155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2546d-a20c-410d-b1ba-9f05b1db392d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:37.963546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2546d-a20c-410d-b1ba-9f05b1db392d admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:37.965469 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2546d-a20c-410d-b1ba-9f05b1db392d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:52:37.965916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2546d-a20c-410d-b1ba-9f05b1db392d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:37.970842 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2546d-a20c-410d-b1ba-9f05b1db392d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:38.008968 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1011/2037] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:52:38.014930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:38.018304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:38.035873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04735f2-b8c8-42e2-996e-79e9a73d045b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:38.043985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0603354-db80-4d43-9fe2-7ea9b020a8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:38.045016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603354-db80-4d43-9fe2-7ea9b020a8cd admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:38.048497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603354-db80-4d43-9fe2-7ea9b020a8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:38.048497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603354-db80-4d43-9fe2-7ea9b020a8cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:38.048497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0603354-db80-4d43-9fe2-7ea9b020a8cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:38.049383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0603354-db80-4d43-9fe2-7ea9b020a8cd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:38.148612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1027/2038] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:38 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:38.164362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2546d-a20c-410d-b1ba-9f05b1db392d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:38.179561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2546d-a20c-410d-b1ba-9f05b1db392d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:38.393902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dadaf-69fc-4e81-a6b5-a8534e53bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:38.395253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dadaf-69fc-4e81-a6b5-a8534e53bb56 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:38.395836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dadaf-69fc-4e81-a6b5-a8534e53bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:38.396429 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dadaf-69fc-4e81-a6b5-a8534e53bb56 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:38.399001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dadaf-69fc-4e81-a6b5-a8534e53bb56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:38.403482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dadaf-69fc-4e81-a6b5-a8534e53bb56 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:38.546779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1012/2039] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:38 2022] POST /identity/v3/users => generated 292 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:38.549764 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0603354-db80-4d43-9fe2-7ea9b020a8cd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:38.575147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0603354-db80-4d43-9fe2-7ea9b020a8cd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:38.584248 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72480876-3a58-45fd-9ef1-cffef383c7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:38.586179 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480876-3a58-45fd-9ef1-cffef383c7dd admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:38.586828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480876-3a58-45fd-9ef1-cffef383c7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:38.587278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480876-3a58-45fd-9ef1-cffef383c7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:38.588335 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72480876-3a58-45fd-9ef1-cffef383c7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:38.591924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72480876-3a58-45fd-9ef1-cffef383c7dd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:38.622764 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1028/2040] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:38.625415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dadaf-69fc-4e81-a6b5-a8534e53bb56 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:38.646651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dadaf-69fc-4e81-a6b5-a8534e53bb56 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:38.653285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f4c29-7ede-4695-ad31-5a691bfd95d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:38.654206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f4c29-7ede-4695-ad31-5a691bfd95d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:38.654673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f4c29-7ede-4695-ad31-5a691bfd95d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:38.655047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f4c29-7ede-4695-ad31-5a691bfd95d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:38.655876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f4c29-7ede-4695-ad31-5a691bfd95d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:38.658046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f4c29-7ede-4695-ad31-5a691bfd95d3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:38.673585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1013/2041] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:38.679552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72480876-3a58-45fd-9ef1-cffef383c7dd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:38.699751 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72480876-3a58-45fd-9ef1-cffef383c7dd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:38.712322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe020d-73c9-437d-87e9-0f1a87b250d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:38.713322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe020d-73c9-437d-87e9-0f1a87b250d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:38.713735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe020d-73c9-437d-87e9-0f1a87b250d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:38.714195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe020d-73c9-437d-87e9-0f1a87b250d3 admin admin] PATH_INFO: `/v3/projects/f2e3e3b06dd24ae5ad6292cd26612a71/users/2dcec9e2f06446c2a3d02aea6dbb5f11/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:38.744034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe020d-73c9-437d-87e9-0f1a87b250d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e3e3b06dd24ae5ad6292cd26612a71, user_id=2dcec9e2f06446c2a3d02aea6dbb5f11, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:38.745059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f4c29-7ede-4695-ad31-5a691bfd95d3 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:38.747891 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-679f4c29-7ede-4695-ad31-5a691bfd95d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:38.751513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe020d-73c9-437d-87e9-0f1a87b250d3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:38.799100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f4c29-7ede-4695-ad31-5a691bfd95d3 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:38.799100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:38.799100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:38.799100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:38.800864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] PATH_INFO: `/v3/projects/71488192c3bf44d69a45994d247a5665` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:38.800864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] RBAC: Authorizing `identity:delete_project(project_id=71488192c3bf44d69a45994d247a5665)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:38.800864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:38.906906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1029/2042] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:38 2022] PUT /identity/v3/projects/f2e3e3b06dd24ae5ad6292cd26612a71/users/2dcec9e2f06446c2a3d02aea6dbb5f11/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:38.909693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e0efd-bce5-42fa-b5b6-4b45943009c7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:38.910261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e0efd-bce5-42fa-b5b6-4b45943009c7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:38.910672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e0efd-bce5-42fa-b5b6-4b45943009c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:38.970043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e85e0efd-bce5-42fa-b5b6-4b45943009c7 None None] MFA Rules not processed for user `2dcec9e2f06446c2a3d02aea6dbb5f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:38.973182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e85e0efd-bce5-42fa-b5b6-4b45943009c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:38.979404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:39.022685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71488192c3bf44d69a45994d247a5665', 'request_id': 'req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:39.068672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:39.099075 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1030/2043] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:52:39.104164 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e0efd-bce5-42fa-b5b6-4b45943009c7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:39.130337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71488192c3bf44d69a45994d247a5665', 'request_id': 'req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:39.137876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71488192c3bf44d69a45994d247a5665', 'request_id': 'req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:39.154636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] The token cache is being invalidate because project 71488192c3bf44d69a45994d247a5665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:39.154636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:39.162326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:39.169303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1014/2044] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:38 2022] DELETE /identity/v3/projects/71488192c3bf44d69a45994d247a5665 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:39.175909 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:39.180582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:39.250538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e0efd-bce5-42fa-b5b6-4b45943009c7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:39.322990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ae06d-48b8-42de-98fa-47ad03e1d92e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:39.348217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:39.348217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:39.348217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:39.348217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] PATH_INFO: `/v3/users/3de599844987433f9dd4cc98aadf9d61` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:39.350403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de599844987433f9dd4cc98aadf9d61)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:39.353895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:39.386053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed4632-80b8-46cf-ad0d-ab2c5fe5a228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:39.386695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed4632-80b8-46cf-ad0d-ab2c5fe5a228 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:39.387481 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed4632-80b8-46cf-ad0d-ab2c5fe5a228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:39.387962 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed4632-80b8-46cf-ad0d-ab2c5fe5a228 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:39.389076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed4632-80b8-46cf-ad0d-ab2c5fe5a228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:39.395168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed4632-80b8-46cf-ad0d-ab2c5fe5a228 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:39.550762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1031/2045] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:39 2022] POST /identity/v3/projects => generated 381 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:52:39.550762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed4632-80b8-46cf-ad0d-ab2c5fe5a228 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:39.560998 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-47ed4632-80b8-46cf-ad0d-ab2c5fe5a228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:39.679385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de599844987433f9dd4cc98aadf9d61'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:39.725426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de599844987433f9dd4cc98aadf9d61'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:39.779026 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de599844987433f9dd4cc98aadf9d61'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:39.783761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed4632-80b8-46cf-ad0d-ab2c5fe5a228 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:39.796623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb057a-9cce-4600-8bf0-d2943fc136cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:39.797227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb057a-9cce-4600-8bf0-d2943fc136cf admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:39.797831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb057a-9cce-4600-8bf0-d2943fc136cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:39.798472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb057a-9cce-4600-8bf0-d2943fc136cf admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:39.799815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb057a-9cce-4600-8bf0-d2943fc136cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:39.804265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb057a-9cce-4600-8bf0-d2943fc136cf admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:39.902222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de599844987433f9dd4cc98aadf9d61'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:39.903434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de599844987433f9dd4cc98aadf9d61'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:39.919440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1015/2046] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:39 2022] DELETE /identity/v3/users/3de599844987433f9dd4cc98aadf9d61 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:39.949707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1032/2047] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:39 2022] POST /identity/v3/users => generated 291 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:39.950659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:39.952687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:40.025586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:40.076508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5d17d-d67d-4443-b881-e060e28be0a9 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:40.085244 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfdf0c-19cb-48ee-a2d2-b529658c0775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:40.087304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfdf0c-19cb-48ee-a2d2-b529658c0775 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:40.087795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfdf0c-19cb-48ee-a2d2-b529658c0775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:40.089719 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfdf0c-19cb-48ee-a2d2-b529658c0775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:40.091075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfdf0c-19cb-48ee-a2d2-b529658c0775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:40.101733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfdf0c-19cb-48ee-a2d2-b529658c0775 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:40.130460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1016/2048] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:40.135387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb057a-9cce-4600-8bf0-d2943fc136cf admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:40.150271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:40.159688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb057a-9cce-4600-8bf0-d2943fc136cf admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:40.167551 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961df2f-5631-49b0-a495-19ed2dcf2d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:40.168709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961df2f-5631-49b0-a495-19ed2dcf2d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:40.169236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961df2f-5631-49b0-a495-19ed2dcf2d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:40.169822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961df2f-5631-49b0-a495-19ed2dcf2d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:40.172221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961df2f-5631-49b0-a495-19ed2dcf2d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:40.179039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961df2f-5631-49b0-a495-19ed2dcf2d17 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:40.215022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1033/2049] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:40.222181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfdf0c-19cb-48ee-a2d2-b529658c0775 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:40.235291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832aa24-9cbd-4524-a884-44d9dbdce40c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:40.235775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832aa24-9cbd-4524-a884-44d9dbdce40c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:40.236252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832aa24-9cbd-4524-a884-44d9dbdce40c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:40.241419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1034/2050] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:52:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:52:40.248474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfdf0c-19cb-48ee-a2d2-b529658c0775 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:40.261531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d11d9-1ac0-4f5f-96ad-7b946cb837dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:40.263744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d11d9-1ac0-4f5f-96ad-7b946cb837dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:40.264877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d11d9-1ac0-4f5f-96ad-7b946cb837dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:40.266221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d11d9-1ac0-4f5f-96ad-7b946cb837dc admin admin] PATH_INFO: `/v3/projects/9096e291fe064b16b16154affdf7a49f/users/f288fe28b7274141ac8643ac84f76681/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:40.295197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068a543-ef58-492e-be25-58958f282431 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:40.297156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068a543-ef58-492e-be25-58958f282431 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:40.297942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068a543-ef58-492e-be25-58958f282431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:40.310169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d11d9-1ac0-4f5f-96ad-7b946cb837dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9096e291fe064b16b16154affdf7a49f, user_id=f288fe28b7274141ac8643ac84f76681, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:40.316579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d11d9-1ac0-4f5f-96ad-7b946cb837dc admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:40.363096 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9068a543-ef58-492e-be25-58958f282431 None None] MFA Rules not processed for user `fb770d4379e74bf79ad4280329c771c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:40.364770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9068a543-ef58-492e-be25-58958f282431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:40.430227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1017/2051] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:40 2022] PUT /identity/v3/projects/9096e291fe064b16b16154affdf7a49f/users/f288fe28b7274141ac8643ac84f76681/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:40.452824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa8f18-2af4-4a6c-a605-36f665747ead None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:40.453111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa8f18-2af4-4a6c-a605-36f665747ead None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:40.453422 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa8f18-2af4-4a6c-a605-36f665747ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:40.521987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-46fa8f18-2af4-4a6c-a605-36f665747ead None None] MFA Rules not processed for user `f288fe28b7274141ac8643ac84f76681`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:40.526646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1035/2052] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:52:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:52:40.529782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068a543-ef58-492e-be25-58958f282431 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:40.532400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9068a543-ef58-492e-be25-58958f282431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:40.533100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-46fa8f18-2af4-4a6c-a605-36f665747ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:40.557369 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068a543-ef58-492e-be25-58958f282431 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:40.646696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:40.648126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:40.649014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1018/2053] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:40.651788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa8f18-2af4-4a6c-a605-36f665747ead None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:40.654178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:40.654625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-46fa8f18-2af4-4a6c-a605-36f665747ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:40.655047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] PATH_INFO: `/v3/users/2b88462a0d8d4e3fb4672a3f1d219c28` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:40.658651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b88462a0d8d4e3fb4672a3f1d219c28)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:40.667073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:40.834234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa8f18-2af4-4a6c-a605-36f665747ead None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:40.840854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f1fb6-a311-4fb2-b0cb-cbcff177afbe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:40.841799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f1fb6-a311-4fb2-b0cb-cbcff177afbe service placement] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:40.842257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f1fb6-a311-4fb2-b0cb-cbcff177afbe service placement] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:40.842763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f1fb6-a311-4fb2-b0cb-cbcff177afbe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:40.868853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f1fb6-a311-4fb2-b0cb-cbcff177afbe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:40.879312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f1fb6-a311-4fb2-b0cb-cbcff177afbe service placement] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:40.894221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1019/2054] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:40.896086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f1fb6-a311-4fb2-b0cb-cbcff177afbe service placement] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:40.898702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-dd3f1fb6-a311-4fb2-b0cb-cbcff177afbe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:40.930256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f1fb6-a311-4fb2-b0cb-cbcff177afbe service placement] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:40.939203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:40.940215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:40.940675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:40.941125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] PATH_INFO: `/v3/projects/e55ea4203d624c3ea06854ec6adf92be` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:40.943693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e55ea4203d624c3ea06854ec6adf92be)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:40.948087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:40.968069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b88462a0d8d4e3fb4672a3f1d219c28'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:40.988554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:40.993915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b88462a0d8d4e3fb4672a3f1d219c28'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:41.006599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b88462a0d8d4e3fb4672a3f1d219c28'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:41.096096 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e55ea4203d624c3ea06854ec6adf92be', 'request_id': 'req-a720e01e-7ba8-416a-87d2-01cbe5743fc5'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:41.118138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b88462a0d8d4e3fb4672a3f1d219c28'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:41.119351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b88462a0d8d4e3fb4672a3f1d219c28'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:41.130260 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1036/2055] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:40 2022] DELETE /identity/v3/users/2b88462a0d8d4e3fb4672a3f1d219c28 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:41.131838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:41.134316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:41.158048 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:41.234439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e55ea4203d624c3ea06854ec6adf92be', 'request_id': 'req-a720e01e-7ba8-416a-87d2-01cbe5743fc5'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:41.247832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e55ea4203d624c3ea06854ec6adf92be', 'request_id': 'req-a720e01e-7ba8-416a-87d2-01cbe5743fc5'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:41.272704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] The token cache is being invalidate because project e55ea4203d624c3ea06854ec6adf92be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:41.273451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:41.274689 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a720e01e-7ba8-416a-87d2-01cbe5743fc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:41.282491 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1020/2056] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:40 2022] DELETE /identity/v3/projects/e55ea4203d624c3ea06854ec6adf92be => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:41.282780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1ed7a-a29e-4c00-94c2-0e6aec3369ef admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:41.288983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69037a-2f6a-47b5-be59-1f4d3499f561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:41.290184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69037a-2f6a-47b5-be59-1f4d3499f561 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:41.290813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69037a-2f6a-47b5-be59-1f4d3499f561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:41.291424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69037a-2f6a-47b5-be59-1f4d3499f561 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:41.292671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159617b6-8228-4e05-bb01-335dbd0b59fc None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:41.292916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69037a-2f6a-47b5-be59-1f4d3499f561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:41.293160 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159617b6-8228-4e05-bb01-335dbd0b59fc None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:41.293650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159617b6-8228-4e05-bb01-335dbd0b59fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:41.296062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69037a-2f6a-47b5-be59-1f4d3499f561 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:41.332779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-159617b6-8228-4e05-bb01-335dbd0b59fc None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:41.334301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-159617b6-8228-4e05-bb01-335dbd0b59fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:41.346867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1021/2057] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:52:41.362068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d5c86-71a3-4c85-b5d1-a8caf3dd0475 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:41.362575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d5c86-71a3-4c85-b5d1-a8caf3dd0475 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:41.363064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d5c86-71a3-4c85-b5d1-a8caf3dd0475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:41.388292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f89d5c86-71a3-4c85-b5d1-a8caf3dd0475 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:41.389871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f89d5c86-71a3-4c85-b5d1-a8caf3dd0475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:41.402619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1022/2058] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:41.407216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d5c86-71a3-4c85-b5d1-a8caf3dd0475 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:41.425299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d5c86-71a3-4c85-b5d1-a8caf3dd0475 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:41.431433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebd704-c189-4f1a-bcec-a37a748a44cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:41.432230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebd704-c189-4f1a-bcec-a37a748a44cd admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:41.432784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebd704-c189-4f1a-bcec-a37a748a44cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:41.433239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebd704-c189-4f1a-bcec-a37a748a44cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:41.434064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebd704-c189-4f1a-bcec-a37a748a44cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:52:41.434545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebd704-c189-4f1a-bcec-a37a748a44cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:41.437514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebd704-c189-4f1a-bcec-a37a748a44cd admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:41.464562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1037/2059] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:41 2022] POST /identity/v3/projects => generated 385 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:41.464822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1023/2060] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:52:41.466118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69037a-2f6a-47b5-be59-1f4d3499f561 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:41.467521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebd704-c189-4f1a-bcec-a37a748a44cd admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:41.468864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ec69037a-2f6a-47b5-be59-1f4d3499f561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:41.479347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebd704-c189-4f1a-bcec-a37a748a44cd admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:41.559196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69037a-2f6a-47b5-be59-1f4d3499f561 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:41.561020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110609e-a071-4f88-bc15-f76c722b6cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:41.562479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110609e-a071-4f88-bc15-f76c722b6cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:41.562977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110609e-a071-4f88-bc15-f76c722b6cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:41.563390 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110609e-a071-4f88-bc15-f76c722b6cf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:41.564201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110609e-a071-4f88-bc15-f76c722b6cf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:41.565152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02da00-2251-41c2-84b7-bef02d35be75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:41.565846 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02da00-2251-41c2-84b7-bef02d35be75 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:41.566125 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02da00-2251-41c2-84b7-bef02d35be75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:41.566400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02da00-2251-41c2-84b7-bef02d35be75 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:41.567384 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110609e-a071-4f88-bc15-f76c722b6cf6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:41.568760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02da00-2251-41c2-84b7-bef02d35be75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:41.572268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02da00-2251-41c2-84b7-bef02d35be75 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:41.726041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1038/2061] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:41 2022] POST /identity/v3/users => generated 299 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:41.728003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02da00-2251-41c2-84b7-bef02d35be75 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:41.731103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8d02da00-2251-41c2-84b7-bef02d35be75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:41.746188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02da00-2251-41c2-84b7-bef02d35be75 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:41.751870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:41.752552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:41.752827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:41.753098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] PATH_INFO: `/v3/projects/a8ada08f998844e997babd510083c205` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:41.754515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ada08f998844e997babd510083c205)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:41.756794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:41.778449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1024/2062] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:41 2022] POST /identity/v3/projects => generated 385 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:41.780346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110609e-a071-4f88-bc15-f76c722b6cf6 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:41.884335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110609e-a071-4f88-bc15-f76c722b6cf6 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:41.893106 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ada08f998844e997babd510083c205', 'request_id': 'req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:41.905415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ada08f998844e997babd510083c205', 'request_id': 'req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:41.975581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:41.976451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5646b89-9411-40a4-9495-d795aab03671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:41.977172 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5646b89-9411-40a4-9495-d795aab03671 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:41.977445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5646b89-9411-40a4-9495-d795aab03671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:41.977711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5646b89-9411-40a4-9495-d795aab03671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:41.978367 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5646b89-9411-40a4-9495-d795aab03671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:41.980210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5646b89-9411-40a4-9495-d795aab03671 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:41.992545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1025/2063] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:41.993566 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5646b89-9411-40a4-9495-d795aab03671 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:42.004607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5646b89-9411-40a4-9495-d795aab03671 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:42.009624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ada08f998844e997babd510083c205', 'request_id': 'req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:42.010484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8c818-f2e2-415d-bb96-343dea26b9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:42.010664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c818-f2e2-415d-bb96-343dea26b9ba admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.010839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c818-f2e2-415d-bb96-343dea26b9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.011117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c818-f2e2-415d-bb96-343dea26b9ba admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.011773 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8c818-f2e2-415d-bb96-343dea26b9ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:42.013760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8c818-f2e2-415d-bb96-343dea26b9ba admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:42.026060 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] The token cache is being invalidate because project a8ada08f998844e997babd510083c205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:52:42.026528 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:42.027916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:42.029497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1039/2064] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:52:41 2022] DELETE /identity/v3/projects/a8ada08f998844e997babd510083c205 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:42.030577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:42.037374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:42.052317 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcf10e-d3ac-4bb6-b453-7ab558bbb69c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:42.058823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff0535-7d33-42a4-ba5b-423bbfee7bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:42.059571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff0535-7d33-42a4-ba5b-423bbfee7bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.059886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff0535-7d33-42a4-ba5b-423bbfee7bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.060263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff0535-7d33-42a4-ba5b-423bbfee7bbe admin admin] PATH_INFO: `/v3/projects/727c1622e1eb4e7790a05d9918f2f182/users/ec14b789c6ce41c588b00a7cd7a11c5e/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.080706 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff0535-7d33-42a4-ba5b-423bbfee7bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=727c1622e1eb4e7790a05d9918f2f182, user_id=ec14b789c6ce41c588b00a7cd7a11c5e, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:42.083561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff0535-7d33-42a4-ba5b-423bbfee7bbe admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:42.150185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1026/2065] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:41 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:42.153104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a035c08-056f-48bd-b2a1-1bfccc929a90 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.153559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a035c08-056f-48bd-b2a1-1bfccc929a90 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.154113 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a035c08-056f-48bd-b2a1-1bfccc929a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.190695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7a035c08-056f-48bd-b2a1-1bfccc929a90 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:42.192885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7a035c08-056f-48bd-b2a1-1bfccc929a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:42.208564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1040/2066] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:42 2022] PUT /identity/v3/projects/727c1622e1eb4e7790a05d9918f2f182/users/ec14b789c6ce41c588b00a7cd7a11c5e/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:42.209980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff0535-7d33-42a4-ba5b-423bbfee7bbe admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:42.212307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b0ff0535-7d33-42a4-ba5b-423bbfee7bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:42.310987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1027/2067] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:42.311824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a035c08-056f-48bd-b2a1-1bfccc929a90 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:42.340058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a035c08-056f-48bd-b2a1-1bfccc929a90 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:42.341308 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff0535-7d33-42a4-ba5b-423bbfee7bbe admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:42.348705 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22201b-5637-4df9-96ca-e38c2020b511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:42.350100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22201b-5637-4df9-96ca-e38c2020b511 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.350100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22201b-5637-4df9-96ca-e38c2020b511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.350488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22201b-5637-4df9-96ca-e38c2020b511 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.351377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22201b-5637-4df9-96ca-e38c2020b511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:42.353222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22201b-5637-4df9-96ca-e38c2020b511 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:42.368547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325c854-28d4-4ade-89d7-0a279c0d85b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:42.368547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325c854-28d4-4ade-89d7-0a279c0d85b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.368547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325c854-28d4-4ade-89d7-0a279c0d85b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.368547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325c854-28d4-4ade-89d7-0a279c0d85b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.369179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325c854-28d4-4ade-89d7-0a279c0d85b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:42.372744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325c854-28d4-4ade-89d7-0a279c0d85b0 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:42.386595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1041/2068] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:42.388772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c572b-7d2b-40d4-9f16-4cd0fcfcedee None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.389028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c572b-7d2b-40d4-9f16-4cd0fcfcedee None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.389961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c572b-7d2b-40d4-9f16-4cd0fcfcedee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.398185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:42.398788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1028/2069] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:42.399735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325c854-28d4-4ade-89d7-0a279c0d85b0 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:42.423499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325c854-28d4-4ade-89d7-0a279c0d85b0 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:42.443983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-fd5c572b-7d2b-40d4-9f16-4cd0fcfcedee None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:42.444426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc74e8-c0ce-4fb8-8971-2eff90ff2cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:42.445257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fd5c572b-7d2b-40d4-9f16-4cd0fcfcedee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:42.445532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc74e8-c0ce-4fb8-8971-2eff90ff2cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.446055 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc74e8-c0ce-4fb8-8971-2eff90ff2cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.447352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc74e8-c0ce-4fb8-8971-2eff90ff2cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.449759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc74e8-c0ce-4fb8-8971-2eff90ff2cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:42.453117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc74e8-c0ce-4fb8-8971-2eff90ff2cc4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:42.460114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1042/2070] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:42.460114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c572b-7d2b-40d4-9f16-4cd0fcfcedee None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:42.473333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c572b-7d2b-40d4-9f16-4cd0fcfcedee None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:42.479922 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35693c4b-e712-473b-95ac-f223b4e26a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:42.481060 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693c4b-e712-473b-95ac-f223b4e26a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.481381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693c4b-e712-473b-95ac-f223b4e26a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.481742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693c4b-e712-473b-95ac-f223b4e26a77 admin admin] PATH_INFO: `/v3/projects/727c1622e1eb4e7790a05d9918f2f182/users/ec14b789c6ce41c588b00a7cd7a11c5e/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.484110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35693c4b-e712-473b-95ac-f223b4e26a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=727c1622e1eb4e7790a05d9918f2f182, user_id=ec14b789c6ce41c588b00a7cd7a11c5e, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:42.487568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35693c4b-e712-473b-95ac-f223b4e26a77 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:42.489094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1029/2071] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:42.490800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc74e8-c0ce-4fb8-8971-2eff90ff2cc4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:42.512016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1043/2072] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:42 2022] PUT /identity/v3/projects/727c1622e1eb4e7790a05d9918f2f182/users/ec14b789c6ce41c588b00a7cd7a11c5e/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:42.513132 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc74e8-c0ce-4fb8-8971-2eff90ff2cc4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:42.514332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35693c4b-e712-473b-95ac-f223b4e26a77 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:42.641272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37315a9-a35b-4619-b475-848173ad739d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:42.643528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37315a9-a35b-4619-b475-848173ad739d admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.643528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37315a9-a35b-4619-b475-848173ad739d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.643528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37315a9-a35b-4619-b475-848173ad739d admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.644332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37315a9-a35b-4619-b475-848173ad739d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:52:42.648016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37315a9-a35b-4619-b475-848173ad739d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:42.653951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35693c4b-e712-473b-95ac-f223b4e26a77 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:42.658612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37315a9-a35b-4619-b475-848173ad739d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:42.666541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3a2c3-0b23-417b-87b5-4d98b86babc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:42.667624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3a2c3-0b23-417b-87b5-4d98b86babc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.668087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3a2c3-0b23-417b-87b5-4d98b86babc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.668533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3a2c3-0b23-417b-87b5-4d98b86babc7 admin admin] PATH_INFO: `/v3/projects/47e7cdc87a124e87bd52e64755793cef/users/923b4a0f8a1746b48f5c1aed8883533e/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.681733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1030/2073] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:52:42.684562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20a30-f732-4cbd-8bac-e6b3277fbc94 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.684562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20a30-f732-4cbd-8bac-e6b3277fbc94 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.684562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20a30-f732-4cbd-8bac-e6b3277fbc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.695874 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3a2c3-0b23-417b-87b5-4d98b86babc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47e7cdc87a124e87bd52e64755793cef, user_id=923b4a0f8a1746b48f5c1aed8883533e, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:42.701343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3a2c3-0b23-417b-87b5-4d98b86babc7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:42.728795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:42.740629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-b8c20a30-f732-4cbd-8bac-e6b3277fbc94 None None] MFA Rules not processed for user `ec14b789c6ce41c588b00a7cd7a11c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:42.742165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b8c20a30-f732-4cbd-8bac-e6b3277fbc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:42.837425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1031/2074] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:42.837425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20a30-f732-4cbd-8bac-e6b3277fbc94 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:42.857471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20a30-f732-4cbd-8bac-e6b3277fbc94 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:42.857990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1044/2075] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:42 2022] PUT /identity/v3/projects/47e7cdc87a124e87bd52e64755793cef/users/923b4a0f8a1746b48f5c1aed8883533e/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:42.861310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3a2c3-0b23-417b-87b5-4d98b86babc7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:42.864554 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c3b3a2c3-0b23-417b-87b5-4d98b86babc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:42.947736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:42.963671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3a2c3-0b23-417b-87b5-4d98b86babc7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:42.967718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f930a-7ec7-49c1-a749-ce229d0a0bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:42.968596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631da4bf-dc9b-44c4-b99b-d5a63da89b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:42.969250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f930a-7ec7-49c1-a749-ce229d0a0bdd admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.969645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631da4bf-dc9b-44c4-b99b-d5a63da89b7c service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:42.969799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f930a-7ec7-49c1-a749-ce229d0a0bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.970098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f930a-7ec7-49c1-a749-ce229d0a0bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.970260 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631da4bf-dc9b-44c4-b99b-d5a63da89b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:42.970572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631da4bf-dc9b-44c4-b99b-d5a63da89b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:42.971139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f930a-7ec7-49c1-a749-ce229d0a0bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:42.973568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-631da4bf-dc9b-44c4-b99b-d5a63da89b7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:42.974006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f930a-7ec7-49c1-a749-ce229d0a0bdd admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:43.057994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631da4bf-dc9b-44c4-b99b-d5a63da89b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:43.059529 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631da4bf-dc9b-44c4-b99b-d5a63da89b7c service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:43.070728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1045/2076] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:43.072339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8addac-ccf4-473b-9b5e-95910c04aa32 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:43.072756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8addac-ccf4-473b-9b5e-95910c04aa32 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:43.073145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8addac-ccf4-473b-9b5e-95910c04aa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:43.117276 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6d8addac-ccf4-473b-9b5e-95910c04aa32 None None] MFA Rules not processed for user `923b4a0f8a1746b48f5c1aed8883533e`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:43.119655 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6d8addac-ccf4-473b-9b5e-95910c04aa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:43.126966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1032/2077] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:42 2022] POST /identity/v3/projects => generated 363 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:43.132200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f930a-7ec7-49c1-a749-ce229d0a0bdd admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:43.219031 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f930a-7ec7-49c1-a749-ce229d0a0bdd admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:43.225584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc01e6c-129b-4b74-ade2-ea5478a22709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:43.226511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc01e6c-129b-4b74-ade2-ea5478a22709 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:43.226869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc01e6c-129b-4b74-ade2-ea5478a22709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:43.227098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc01e6c-129b-4b74-ade2-ea5478a22709 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:43.227871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc01e6c-129b-4b74-ade2-ea5478a22709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:43.229817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc01e6c-129b-4b74-ade2-ea5478a22709 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:43.261943 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1046/2078] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:43.279778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8addac-ccf4-473b-9b5e-95910c04aa32 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:43.282780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6d8addac-ccf4-473b-9b5e-95910c04aa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:43.303527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8addac-ccf4-473b-9b5e-95910c04aa32 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:43.370322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1033/2079] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:43 2022] POST /identity/v3/users => generated 282 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:52:43.375473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc01e6c-129b-4b74-ade2-ea5478a22709 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:43.388300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc01e6c-129b-4b74-ade2-ea5478a22709 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:43.397991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27a620-35c0-4e4d-a3ed-582bcc78fcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:43.399237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27a620-35c0-4e4d-a3ed-582bcc78fcc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:43.399810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27a620-35c0-4e4d-a3ed-582bcc78fcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:43.400381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27a620-35c0-4e4d-a3ed-582bcc78fcc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:43.401665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27a620-35c0-4e4d-a3ed-582bcc78fcc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:43.405046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27a620-35c0-4e4d-a3ed-582bcc78fcc8 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:43.406771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cd319-f9a9-4958-a080-c1c9aa09852e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:43.407847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cd319-f9a9-4958-a080-c1c9aa09852e service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:43.408358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cd319-f9a9-4958-a080-c1c9aa09852e service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:43.408855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cd319-f9a9-4958-a080-c1c9aa09852e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:43.410583 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:43.429617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1034/2080] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:52:43.442277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27a620-35c0-4e4d-a3ed-582bcc78fcc8 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:43.454918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27a620-35c0-4e4d-a3ed-582bcc78fcc8 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:43.460782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf0c12-a40a-428e-b4c6-42c31aa1cf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:43.461497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf0c12-a40a-428e-b4c6-42c31aa1cf67 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:43.461801 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf0c12-a40a-428e-b4c6-42c31aa1cf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:43.462248 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf0c12-a40a-428e-b4c6-42c31aa1cf67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:43.463044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf0c12-a40a-428e-b4c6-42c31aa1cf67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:43.465051 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf0c12-a40a-428e-b4c6-42c31aa1cf67 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:43.470724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:43.478530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1035/2081] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:43.485140 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf0c12-a40a-428e-b4c6-42c31aa1cf67 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:43.495009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638cd319-f9a9-4958-a080-c1c9aa09852e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:43.496153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf0c12-a40a-428e-b4c6-42c31aa1cf67 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:43.496678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638cd319-f9a9-4958-a080-c1c9aa09852e service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:43.502028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d26f3-9769-42c2-b389-c57c6dac67b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:43.503765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d26f3-9769-42c2-b389-c57c6dac67b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:43.503765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d26f3-9769-42c2-b389-c57c6dac67b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:43.503765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d26f3-9769-42c2-b389-c57c6dac67b6 admin admin] PATH_INFO: `/v3/projects/fae859ae715742b48c197b304d9f7fc6/users/35ca014c06ac488e861e2d9fd2827179/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:43.513887 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1047/2082] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:52:43.526390 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d26f3-9769-42c2-b389-c57c6dac67b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae859ae715742b48c197b304d9f7fc6, user_id=35ca014c06ac488e861e2d9fd2827179, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:43.529718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d26f3-9769-42c2-b389-c57c6dac67b6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:43.678472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1036/2083] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:43 2022] PUT /identity/v3/projects/fae859ae715742b48c197b304d9f7fc6/users/35ca014c06ac488e861e2d9fd2827179/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:52:43.680983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6193040-10bf-4492-8e61-dd63e97aa8b2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:43.681517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6193040-10bf-4492-8e61-dd63e97aa8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:43.682012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6193040-10bf-4492-8e61-dd63e97aa8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:43.746931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e6193040-10bf-4492-8e61-dd63e97aa8b2 None None] MFA Rules not processed for user `35ca014c06ac488e861e2d9fd2827179`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:43.749971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e6193040-10bf-4492-8e61-dd63e97aa8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:43.849534 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1048/2084] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:43.854666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d26f3-9769-42c2-b389-c57c6dac67b6 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:43.986725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d26f3-9769-42c2-b389-c57c6dac67b6 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:44.007320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34029-ebc8-4368-bd12-08160c4955f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:44.007320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34029-ebc8-4368-bd12-08160c4955f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:44.007320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34029-ebc8-4368-bd12-08160c4955f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:44.007320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34029-ebc8-4368-bd12-08160c4955f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:44.007888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e34029-ebc8-4368-bd12-08160c4955f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:44.007888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e34029-ebc8-4368-bd12-08160c4955f5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:44.163960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1037/2085] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:52:43 2022] POST /identity/v3/projects => generated 365 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:52:44.168571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6193040-10bf-4492-8e61-dd63e97aa8b2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:44.185538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6193040-10bf-4492-8e61-dd63e97aa8b2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:44.275189 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfd151-df2a-4fb9-8abf-f5c18a8e9727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:44.275948 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfd151-df2a-4fb9-8abf-f5c18a8e9727 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:44.276413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfd151-df2a-4fb9-8abf-f5c18a8e9727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:44.276753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfd151-df2a-4fb9-8abf-f5c18a8e9727 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:44.277809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfd151-df2a-4fb9-8abf-f5c18a8e9727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:44.280429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfd151-df2a-4fb9-8abf-f5c18a8e9727 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:44.413487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1049/2086] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:52:44 2022] POST /identity/v3/users => generated 289 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:52:44.425397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34029-ebc8-4368-bd12-08160c4955f5 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:44.437623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34029-ebc8-4368-bd12-08160c4955f5 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:44.442434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781d9e7-fec4-4e03-8446-c7a7eb01d6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:44.443084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781d9e7-fec4-4e03-8446-c7a7eb01d6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:44.443368 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781d9e7-fec4-4e03-8446-c7a7eb01d6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:44.443641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781d9e7-fec4-4e03-8446-c7a7eb01d6f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:44.444317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781d9e7-fec4-4e03-8446-c7a7eb01d6f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:44.446208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781d9e7-fec4-4e03-8446-c7a7eb01d6f4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:44.460782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1038/2087] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:52:44.470373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfd151-df2a-4fb9-8abf-f5c18a8e9727 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:44.483246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfd151-df2a-4fb9-8abf-f5c18a8e9727 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:44.489015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f67d5-e5d7-4581-a1af-4d632cfedeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:44.494485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f67d5-e5d7-4581-a1af-4d632cfedeac admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:44.494485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f67d5-e5d7-4581-a1af-4d632cfedeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:44.497790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f67d5-e5d7-4581-a1af-4d632cfedeac admin admin] PATH_INFO: `/v3/projects/89b235e76ca44aa78ddfb001e654c95b/users/62073fc54ec8489b839d52d03a1f918c/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:44.520225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f67d5-e5d7-4581-a1af-4d632cfedeac admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b235e76ca44aa78ddfb001e654c95b, user_id=62073fc54ec8489b839d52d03a1f918c, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:44.526154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f67d5-e5d7-4581-a1af-4d632cfedeac admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:44.605376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1050/2088] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:44 2022] PUT /identity/v3/projects/89b235e76ca44aa78ddfb001e654c95b/users/62073fc54ec8489b839d52d03a1f918c/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:44.610913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781d9e7-fec4-4e03-8446-c7a7eb01d6f4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:44.622343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781d9e7-fec4-4e03-8446-c7a7eb01d6f4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:44.695338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba258711-258c-47d4-bb5f-04ab74f1b3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:44.695962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba258711-258c-47d4-bb5f-04ab74f1b3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:44.696235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba258711-258c-47d4-bb5f-04ab74f1b3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:44.696512 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba258711-258c-47d4-bb5f-04ab74f1b3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:44.698053 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba258711-258c-47d4-bb5f-04ab74f1b3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:44.700101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba258711-258c-47d4-bb5f-04ab74f1b3d2 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:44.712408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1039/2089] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:52:44.717416 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f67d5-e5d7-4581-a1af-4d632cfedeac admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:44.737663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f67d5-e5d7-4581-a1af-4d632cfedeac admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:44.745715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc10f7d-cc75-400d-876b-7baf79842505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:44.746938 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc10f7d-cc75-400d-876b-7baf79842505 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:44.747447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc10f7d-cc75-400d-876b-7baf79842505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:44.748036 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc10f7d-cc75-400d-876b-7baf79842505 admin admin] PATH_INFO: `/v3/projects/89b235e76ca44aa78ddfb001e654c95b/users/62073fc54ec8489b839d52d03a1f918c/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:44.751680 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc10f7d-cc75-400d-876b-7baf79842505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b235e76ca44aa78ddfb001e654c95b, user_id=62073fc54ec8489b839d52d03a1f918c, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:44.754733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc10f7d-cc75-400d-876b-7baf79842505 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:44.770909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1051/2090] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:52:44 2022] PUT /identity/v3/projects/89b235e76ca44aa78ddfb001e654c95b/users/62073fc54ec8489b839d52d03a1f918c/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:44.792362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6b597-19e9-4835-b95c-e854d172daa8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:44.792906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6b597-19e9-4835-b95c-e854d172daa8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:44.793855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6b597-19e9-4835-b95c-e854d172daa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:44.845790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-2ef6b597-19e9-4835-b95c-e854d172daa8 None None] MFA Rules not processed for user `62073fc54ec8489b839d52d03a1f918c`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:44.847884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2ef6b597-19e9-4835-b95c-e854d172daa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:44.878039 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:44.963515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1040/2091] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:44.983230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc10f7d-cc75-400d-876b-7baf79842505 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:45.034289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:45.088855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc10f7d-cc75-400d-876b-7baf79842505 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:45.115014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fd5d8-2431-4b04-be92-cf8f6eb919d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:45.116073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fd5d8-2431-4b04-be92-cf8f6eb919d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:45.116571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fd5d8-2431-4b04-be92-cf8f6eb919d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:45.117149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fd5d8-2431-4b04-be92-cf8f6eb919d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:45.139313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fd5d8-2431-4b04-be92-cf8f6eb919d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:45.141820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fd5d8-2431-4b04-be92-cf8f6eb919d2 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:45.166458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1052/2092] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:45.460606 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6b597-19e9-4835-b95c-e854d172daa8 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:45.477402 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6b597-19e9-4835-b95c-e854d172daa8 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:45.483209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a1d06-32a0-4a53-b1c4-3625407c923e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:45.484536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1d06-32a0-4a53-b1c4-3625407c923e service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:45.484918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1d06-32a0-4a53-b1c4-3625407c923e service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:45.485217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a1d06-32a0-4a53-b1c4-3625407c923e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:45.572829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a1d06-32a0-4a53-b1c4-3625407c923e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:45.574980 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a1d06-32a0-4a53-b1c4-3625407c923e service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:45.587877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1041/2093] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:46.343675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fd5d8-2431-4b04-be92-cf8f6eb919d2 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:46.364726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fd5d8-2431-4b04-be92-cf8f6eb919d2 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:46.378305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62011c3e-a7f9-41ca-979d-734ab590a725 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:46.379403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62011c3e-a7f9-41ca-979d-734ab590a725 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:46.379878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62011c3e-a7f9-41ca-979d-734ab590a725 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:46.380621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62011c3e-a7f9-41ca-979d-734ab590a725 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:46.384139 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-62011c3e-a7f9-41ca-979d-734ab590a725 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:46.497036 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62011c3e-a7f9-41ca-979d-734ab590a725 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:46.499189 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62011c3e-a7f9-41ca-979d-734ab590a725 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:46.511407 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1053/2094] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:49.248304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a1d06-32a0-4a53-b1c4-3625407c923e service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:49.258609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a1d06-32a0-4a53-b1c4-3625407c923e service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:49.262593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca41b8d-7ba7-4a85-b117-73680226cee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:49.263218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca41b8d-7ba7-4a85-b117-73680226cee0 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:49.263571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca41b8d-7ba7-4a85-b117-73680226cee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:49.263857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca41b8d-7ba7-4a85-b117-73680226cee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:49.341347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca41b8d-7ba7-4a85-b117-73680226cee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:49.343809 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca41b8d-7ba7-4a85-b117-73680226cee0 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:49.355394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1042/2095] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:52:56.505195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=44 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:58.843916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62011c3e-a7f9-41ca-979d-734ab590a725 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:58.925513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62011c3e-a7f9-41ca-979d-734ab590a725 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:58.934404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:58.935089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:58.935906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:58.936181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] PATH_INFO: `/v3/users/923b4a0f8a1746b48f5c1aed8883533e` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:58.937334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=923b4a0f8a1746b48f5c1aed8883533e)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:58.939436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:59.254069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '923b4a0f8a1746b48f5c1aed8883533e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:59.270359 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '923b4a0f8a1746b48f5c1aed8883533e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:59.278845 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '923b4a0f8a1746b48f5c1aed8883533e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:59.349880 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=77 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:59.637170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '923b4a0f8a1746b48f5c1aed8883533e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:59.639090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '923b4a0f8a1746b48f5c1aed8883533e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:52:59.653012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1054/2096] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:58 2022] DELETE /identity/v3/users/923b4a0f8a1746b48f5c1aed8883533e => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:52:59.662581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67623e5b-e3b8-401c-ac4e-9ce0550abd52 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:59.665421 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67623e5b-e3b8-401c-ac4e-9ce0550abd52 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:59.666614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67623e5b-e3b8-401c-ac4e-9ce0550abd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:59.679617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:59.682688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:59.695011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-67623e5b-e3b8-401c-ac4e-9ce0550abd52 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:52:59.696304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-67623e5b-e3b8-401c-ac4e-9ce0550abd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:59.800866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924879e-eb15-4d0c-893f-6fcf5af78ab8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:59.816395 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:59.819393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:59.819393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:59.819393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] PATH_INFO: `/v3/users/2dcec9e2f06446c2a3d02aea6dbb5f11` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:59.821292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dcec9e2f06446c2a3d02aea6dbb5f11)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:59.825878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:59.828964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1043/2097] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:52:59.845683 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67623e5b-e3b8-401c-ac4e-9ce0550abd52 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:52:59.847754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-67623e5b-e3b8-401c-ac4e-9ce0550abd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:52:59.874319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:52:59.945241 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67623e5b-e3b8-401c-ac4e-9ce0550abd52 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:52:59.952070 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61a0c8-d684-41f2-95b0-5897b1ebc57c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:52:59.952763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61a0c8-d684-41f2-95b0-5897b1ebc57c service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:52:59.953043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61a0c8-d684-41f2-95b0-5897b1ebc57c service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:52:59.953322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61a0c8-d684-41f2-95b0-5897b1ebc57c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:52:59.964983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61a0c8-d684-41f2-95b0-5897b1ebc57c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:52:59.968116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61a0c8-d684-41f2-95b0-5897b1ebc57c service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:52:59.979922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1044/2098] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:53:00.226977 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dcec9e2f06446c2a3d02aea6dbb5f11'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:00.241642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dcec9e2f06446c2a3d02aea6dbb5f11'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:00.249968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dcec9e2f06446c2a3d02aea6dbb5f11'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:00.363358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dcec9e2f06446c2a3d02aea6dbb5f11'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:00.364162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dcec9e2f06446c2a3d02aea6dbb5f11'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:00.373205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1055/2099] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:52:59 2022] DELETE /identity/v3/users/2dcec9e2f06446c2a3d02aea6dbb5f11 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:00.379917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfb0e-c021-400e-84ac-cba5bda3cb8e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:00.380775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfb0e-c021-400e-84ac-cba5bda3cb8e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:00.380932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfb0e-c021-400e-84ac-cba5bda3cb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:00.410000 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d24dfb0e-c021-400e-84ac-cba5bda3cb8e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:00.411431 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d24dfb0e-c021-400e-84ac-cba5bda3cb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:00.526096 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1045/2100] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:53:00.536726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:00.540287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:00.564737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dfb0e-c021-400e-84ac-cba5bda3cb8e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:00.566288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d24dfb0e-c021-400e-84ac-cba5bda3cb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:00.581108 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dfb0e-c021-400e-84ac-cba5bda3cb8e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:00.596125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:00.597060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:00.597344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:00.597618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] PATH_INFO: `/v3/projects/47e7cdc87a124e87bd52e64755793cef` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:00.598971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=47e7cdc87a124e87bd52e64755793cef)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:00.601083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:00.637605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80299a1-4455-41bd-b7d6-9e07445082f8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:00.646722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfac529-31b8-4be3-b3b3-4dc3a5f0ccba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:00.646722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfac529-31b8-4be3-b3b3-4dc3a5f0ccba service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:00.646722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfac529-31b8-4be3-b3b3-4dc3a5f0ccba service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:00.646722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfac529-31b8-4be3-b3b3-4dc3a5f0ccba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:00.656432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfac529-31b8-4be3-b3b3-4dc3a5f0ccba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:00.658064 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfac529-31b8-4be3-b3b3-4dc3a5f0ccba service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:00.670175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1056/2101] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:00.802048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:00.809995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47e7cdc87a124e87bd52e64755793cef', 'request_id': 'req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:00.905908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47e7cdc87a124e87bd52e64755793cef', 'request_id': 'req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:00.915800 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47e7cdc87a124e87bd52e64755793cef', 'request_id': 'req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:00.934528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] The token cache is being invalidate because project 47e7cdc87a124e87bd52e64755793cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:00.935010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:00.937709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2a745087-0ee9-4b2d-9f4d-6b6eda9caa8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:00.940508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1046/2102] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:00 2022] DELETE /identity/v3/projects/47e7cdc87a124e87bd52e64755793cef => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:00.948690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900a8e5-875f-47a8-a63d-df0d9cb373d3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:00.949430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900a8e5-875f-47a8-a63d-df0d9cb373d3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:00.949430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900a8e5-875f-47a8-a63d-df0d9cb373d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:00.990664 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8900a8e5-875f-47a8-a63d-df0d9cb373d3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:00.993768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8900a8e5-875f-47a8-a63d-df0d9cb373d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:01.101651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1057/2103] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:01.109984 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1cf7d-3a85-4277-968a-1bd52d9c056b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:01.110268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1cf7d-3a85-4277-968a-1bd52d9c056b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:01.110583 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1cf7d-3a85-4277-968a-1bd52d9c056b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:01.157778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7ca1cf7d-3a85-4277-968a-1bd52d9c056b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:01.160810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7ca1cf7d-3a85-4277-968a-1bd52d9c056b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:01.179419 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1047/2104] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:01.183511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900a8e5-875f-47a8-a63d-df0d9cb373d3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:01.202210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900a8e5-875f-47a8-a63d-df0d9cb373d3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:01.204205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4298b-6857-4474-bd12-e77c3eaed01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:01.205236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4298b-6857-4474-bd12-e77c3eaed01f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:01.205518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4298b-6857-4474-bd12-e77c3eaed01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:01.206589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4298b-6857-4474-bd12-e77c3eaed01f admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:01.207547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4298b-6857-4474-bd12-e77c3eaed01f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:53:01.208024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4298b-6857-4474-bd12-e77c3eaed01f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:01.209876 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4298b-6857-4474-bd12-e77c3eaed01f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:01.231291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1058/2105] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:53:01.232311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1cf7d-3a85-4277-968a-1bd52d9c056b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:01.243599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1cf7d-3a85-4277-968a-1bd52d9c056b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:01.253998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7d09e-46b4-4f15-9571-7be78e179c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:01.259376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7d09e-46b4-4f15-9571-7be78e179c5c admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:01.259376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7d09e-46b4-4f15-9571-7be78e179c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:01.259376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7d09e-46b4-4f15-9571-7be78e179c5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:01.259376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc7d09e-46b4-4f15-9571-7be78e179c5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:01.261708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc7d09e-46b4-4f15-9571-7be78e179c5c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:01.396279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4298b-6857-4474-bd12-e77c3eaed01f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:01.400568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5fa4298b-6857-4474-bd12-e77c3eaed01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:01.404910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1048/2106] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:01 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:01.405721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7d09e-46b4-4f15-9571-7be78e179c5c admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:01.420458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4298b-6857-4474-bd12-e77c3eaed01f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:01.487028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:01.506180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7d09e-46b4-4f15-9571-7be78e179c5c admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:01.515607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0065038-c983-415d-9744-b5f6a1a3eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:01.518645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0065038-c983-415d-9744-b5f6a1a3eea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:01.518645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0065038-c983-415d-9744-b5f6a1a3eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:01.518645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0065038-c983-415d-9744-b5f6a1a3eea6 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:01.518911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0065038-c983-415d-9744-b5f6a1a3eea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:01.530986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0065038-c983-415d-9744-b5f6a1a3eea6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:01.532909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:01.533619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:01.533902 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:01.534181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] PATH_INFO: `/v3/projects/f2e3e3b06dd24ae5ad6292cd26612a71` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:01.535549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2e3e3b06dd24ae5ad6292cd26612a71)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:01.537719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:01.644634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1049/2107] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:01 2022] POST /identity/v3/users => generated 287 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:01.649848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0065038-c983-415d-9744-b5f6a1a3eea6 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:01.660884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0065038-c983-415d-9744-b5f6a1a3eea6 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:01.678388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afe432-b4bc-49ca-8c78-0c6f012e8ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:01.678388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afe432-b4bc-49ca-8c78-0c6f012e8ead admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:01.678388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afe432-b4bc-49ca-8c78-0c6f012e8ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:01.678388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afe432-b4bc-49ca-8c78-0c6f012e8ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:01.679340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afe432-b4bc-49ca-8c78-0c6f012e8ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:01.679340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afe432-b4bc-49ca-8c78-0c6f012e8ead admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:01.681155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1050/2108] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:53:01.687976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afe432-b4bc-49ca-8c78-0c6f012e8ead admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:01.718725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afe432-b4bc-49ca-8c78-0c6f012e8ead admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:01.718725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ede59-fca8-4cf3-8732-ee9dbb64c45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:01.718725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ede59-fca8-4cf3-8732-ee9dbb64c45d admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:01.718725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ede59-fca8-4cf3-8732-ee9dbb64c45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:01.719321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ede59-fca8-4cf3-8732-ee9dbb64c45d admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:01.719321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ede59-fca8-4cf3-8732-ee9dbb64c45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:01.719321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ede59-fca8-4cf3-8732-ee9dbb64c45d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:01.742833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1051/2109] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:01.743327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2e3e3b06dd24ae5ad6292cd26612a71', 'request_id': 'req-e4c664ea-155c-431d-8fb3-fce34df31a29'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:01.743327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2e3e3b06dd24ae5ad6292cd26612a71', 'request_id': 'req-e4c664ea-155c-431d-8fb3-fce34df31a29'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:01.746465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ede59-fca8-4cf3-8732-ee9dbb64c45d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:01.759741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ede59-fca8-4cf3-8732-ee9dbb64c45d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:01.822261 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca8a7e-2dbc-45a5-be67-3cb41046c565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:01.822999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8a7e-2dbc-45a5-be67-3cb41046c565 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:01.823278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8a7e-2dbc-45a5-be67-3cb41046c565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:01.823554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8a7e-2dbc-45a5-be67-3cb41046c565 admin admin] PATH_INFO: `/v3/projects/87299c60c42b43e7bf7e8f92f30ae87d/users/74d57012e9a04960b6c5dde2243dafbc/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:01.836861 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:01.841147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca8a7e-2dbc-45a5-be67-3cb41046c565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87299c60c42b43e7bf7e8f92f30ae87d, user_id=74d57012e9a04960b6c5dde2243dafbc, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:01.844923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca8a7e-2dbc-45a5-be67-3cb41046c565 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:01.849525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2e3e3b06dd24ae5ad6292cd26612a71', 'request_id': 'req-e4c664ea-155c-431d-8fb3-fce34df31a29'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:01.862337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] The token cache is being invalidate because project f2e3e3b06dd24ae5ad6292cd26612a71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:01.862670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:01.863356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:01.864987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1059/2110] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:01 2022] DELETE /identity/v3/projects/f2e3e3b06dd24ae5ad6292cd26612a71 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:01.869162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:01.870558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:01.884059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c664ea-155c-431d-8fb3-fce34df31a29 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:01.890019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:01.890725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:01.891002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:01.891271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] PATH_INFO: `/v3/users/f288fe28b7274141ac8643ac84f76681` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:01.892444 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f288fe28b7274141ac8643ac84f76681)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:01.894443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:02.003970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1052/2111] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:01 2022] PUT /identity/v3/projects/87299c60c42b43e7bf7e8f92f30ae87d/users/74d57012e9a04960b6c5dde2243dafbc/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:02.021006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cfaf-c90e-42eb-a9b2-35e98e93be72 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:02.021287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cfaf-c90e-42eb-a9b2-35e98e93be72 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:02.021559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cfaf-c90e-42eb-a9b2-35e98e93be72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:02.041897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7092cfaf-c90e-42eb-a9b2-35e98e93be72 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:02.045211 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7092cfaf-c90e-42eb-a9b2-35e98e93be72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:02.103317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1053/2112] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:53:02.113722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092cfaf-c90e-42eb-a9b2-35e98e93be72 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:02.115168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7092cfaf-c90e-42eb-a9b2-35e98e93be72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:02.146606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f288fe28b7274141ac8643ac84f76681'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:02.164027 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f288fe28b7274141ac8643ac84f76681'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:02.172501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f288fe28b7274141ac8643ac84f76681'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:02.182184 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092cfaf-c90e-42eb-a9b2-35e98e93be72 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:02.186559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d18394-4222-45fc-b4ad-68260faecc40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:02.187173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d18394-4222-45fc-b4ad-68260faecc40 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:02.187630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d18394-4222-45fc-b4ad-68260faecc40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:02.187731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d18394-4222-45fc-b4ad-68260faecc40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:02.197347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d18394-4222-45fc-b4ad-68260faecc40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:02.198826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d18394-4222-45fc-b4ad-68260faecc40 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:02.208879 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1054/2113] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:53:02.254103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f288fe28b7274141ac8643ac84f76681'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:02.254911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f288fe28b7274141ac8643ac84f76681'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:02.262383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1060/2114] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:01 2022] DELETE /identity/v3/users/f288fe28b7274141ac8643ac84f76681 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:02.770831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d18394-4222-45fc-b4ad-68260faecc40 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:02.789889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d18394-4222-45fc-b4ad-68260faecc40 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:02.880231 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:02.881098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:02.881380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:02.881830 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] PATH_INFO: `/v3/projects/9096e291fe064b16b16154affdf7a49f` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:02.883257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9096e291fe064b16b16154affdf7a49f)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:02.885235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:02.983887 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:02.984574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:03.002129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d908d-7dfa-4bea-ba7e-6456d601aabf admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:03.008379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a8aec-d352-4c25-a7fc-be7be294a749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:03.009185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a8aec-d352-4c25-a7fc-be7be294a749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:03.009619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a8aec-d352-4c25-a7fc-be7be294a749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:03.010002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a8aec-d352-4c25-a7fc-be7be294a749 admin admin] PATH_INFO: `/v3/users/35ca014c06ac488e861e2d9fd2827179` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:03.011809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a8aec-d352-4c25-a7fc-be7be294a749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35ca014c06ac488e861e2d9fd2827179)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:03.020004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a8aec-d352-4c25-a7fc-be7be294a749 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:03.024794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:03.037759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9096e291fe064b16b16154affdf7a49f', 'request_id': 'req-0fe43a65-8e1e-4069-b8b7-64135b04aae0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.168808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9096e291fe064b16b16154affdf7a49f', 'request_id': 'req-0fe43a65-8e1e-4069-b8b7-64135b04aae0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.170582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=77,keystone:DELETE=21,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:03.179824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9096e291fe064b16b16154affdf7a49f', 'request_id': 'req-0fe43a65-8e1e-4069-b8b7-64135b04aae0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.193532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] The token cache is being invalidate because project 9096e291fe064b16b16154affdf7a49f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:03.193869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.194551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.196251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1055/2115] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:02 2022] DELETE /identity/v3/projects/9096e291fe064b16b16154affdf7a49f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:03.200376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:03.203533 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:03.288503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe43a65-8e1e-4069-b8b7-64135b04aae0 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:03.297603 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3424df-ac27-4619-945e-58d32e1be2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:03.298279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3424df-ac27-4619-945e-58d32e1be2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:03.298599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3424df-ac27-4619-945e-58d32e1be2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:03.298874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3424df-ac27-4619-945e-58d32e1be2a3 admin admin] PATH_INFO: `/v3/users/ec14b789c6ce41c588b00a7cd7a11c5e` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:03.300409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3424df-ac27-4619-945e-58d32e1be2a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec14b789c6ce41c588b00a7cd7a11c5e)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:03.302698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3424df-ac27-4619-945e-58d32e1be2a3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:03.304915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-342a8aec-d352-4c25-a7fc-be7be294a749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35ca014c06ac488e861e2d9fd2827179'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.349241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-342a8aec-d352-4c25-a7fc-be7be294a749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35ca014c06ac488e861e2d9fd2827179'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.366776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-342a8aec-d352-4c25-a7fc-be7be294a749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35ca014c06ac488e861e2d9fd2827179'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.468761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-342a8aec-d352-4c25-a7fc-be7be294a749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35ca014c06ac488e861e2d9fd2827179'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.470243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-342a8aec-d352-4c25-a7fc-be7be294a749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35ca014c06ac488e861e2d9fd2827179'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.482786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1061/2116] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:02 2022] DELETE /identity/v3/users/35ca014c06ac488e861e2d9fd2827179 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:03.489599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6d88e-c053-4211-a946-53b894f964c0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:03.490078 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6d88e-c053-4211-a946-53b894f964c0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:03.490591 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6d88e-c053-4211-a946-53b894f964c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:03.515482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-add6d88e-c053-4211-a946-53b894f964c0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:03.516777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-add6d88e-c053-4211-a946-53b894f964c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:03.556767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6a3424df-ac27-4619-945e-58d32e1be2a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec14b789c6ce41c588b00a7cd7a11c5e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.583851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6a3424df-ac27-4619-945e-58d32e1be2a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec14b789c6ce41c588b00a7cd7a11c5e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.602534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6a3424df-ac27-4619-945e-58d32e1be2a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec14b789c6ce41c588b00a7cd7a11c5e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.636671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1062/2117] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:53:03.648039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6d88e-c053-4211-a946-53b894f964c0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:03.650198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-add6d88e-c053-4211-a946-53b894f964c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:03.717405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6a3424df-ac27-4619-945e-58d32e1be2a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec14b789c6ce41c588b00a7cd7a11c5e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.721903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6a3424df-ac27-4619-945e-58d32e1be2a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec14b789c6ce41c588b00a7cd7a11c5e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:03.737571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1056/2118] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:03 2022] DELETE /identity/v3/users/ec14b789c6ce41c588b00a7cd7a11c5e => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:03.803205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6d88e-c053-4211-a946-53b894f964c0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:03.903282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766300b8-7389-4569-a8b8-d5fb94ba8222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:03.904941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766300b8-7389-4569-a8b8-d5fb94ba8222 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:03.905459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766300b8-7389-4569-a8b8-d5fb94ba8222 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:03.905947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766300b8-7389-4569-a8b8-d5fb94ba8222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:03.909096 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-766300b8-7389-4569-a8b8-d5fb94ba8222 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:03.939059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:04.023912 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766300b8-7389-4569-a8b8-d5fb94ba8222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:04.030254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766300b8-7389-4569-a8b8-d5fb94ba8222 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:04.041196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1063/2119] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:53:04.466873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046d93e-1e72-4aec-97d7-674c139b0da3 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:04.467659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046d93e-1e72-4aec-97d7-674c139b0da3 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:04.467659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046d93e-1e72-4aec-97d7-674c139b0da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:04.496124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766300b8-7389-4569-a8b8-d5fb94ba8222 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:04.504413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-766300b8-7389-4569-a8b8-d5fb94ba8222 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:04.510171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-8046d93e-1e72-4aec-97d7-674c139b0da3 None None] MFA Rules not processed for user `74d57012e9a04960b6c5dde2243dafbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:04.512133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8046d93e-1e72-4aec-97d7-674c139b0da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:04.559279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766300b8-7389-4569-a8b8-d5fb94ba8222 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:04.569524 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:04.570233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:04.570663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:04.570950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] PATH_INFO: `/v3/projects/727c1622e1eb4e7790a05d9918f2f182` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:04.572565 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] RBAC: Authorizing `identity:delete_project(project_id=727c1622e1eb4e7790a05d9918f2f182)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:04.574960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:04.636811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:04.696990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1057/2120] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:04.713278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046d93e-1e72-4aec-97d7-674c139b0da3 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:04.715827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8046d93e-1e72-4aec-97d7-674c139b0da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:04.801063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '727c1622e1eb4e7790a05d9918f2f182', 'request_id': 'req-eefffdd4-ce75-477b-95a7-cadcfe1b974a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:04.808412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '727c1622e1eb4e7790a05d9918f2f182', 'request_id': 'req-eefffdd4-ce75-477b-95a7-cadcfe1b974a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:04.822102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046d93e-1e72-4aec-97d7-674c139b0da3 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:04.887570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d15b91-4948-4404-a926-5df59a5a71c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:04.888262 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15b91-4948-4404-a926-5df59a5a71c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:04.888621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15b91-4948-4404-a926-5df59a5a71c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:04.888813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15b91-4948-4404-a926-5df59a5a71c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:04.914108 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '727c1622e1eb4e7790a05d9918f2f182', 'request_id': 'req-eefffdd4-ce75-477b-95a7-cadcfe1b974a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:04.935025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] The token cache is being invalidate because project 727c1622e1eb4e7790a05d9918f2f182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:04.935582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:04.937317 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:04.949460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1064/2121] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:04 2022] DELETE /identity/v3/projects/727c1622e1eb4e7790a05d9918f2f182 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:04.960437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:04.966677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d15b91-4948-4404-a926-5df59a5a71c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:04.968653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:04.968790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d15b91-4948-4404-a926-5df59a5a71c6 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:04.980567 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1058/2122] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:04.983400 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1f6c6-8923-47bb-a871-2d5d493b41e4 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:04.983982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1f6c6-8923-47bb-a871-2d5d493b41e4 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:04.984266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1f6c6-8923-47bb-a871-2d5d493b41e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:05.011181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-50e1f6c6-8923-47bb-a871-2d5d493b41e4 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:05.012751 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-50e1f6c6-8923-47bb-a871-2d5d493b41e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:05.111231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefffdd4-ce75-477b-95a7-cadcfe1b974a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:05.116278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1059/2123] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:05.117307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:05.117976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:05.118259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:05.118695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] PATH_INFO: `/v3/projects/fae859ae715742b48c197b304d9f7fc6` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:05.120360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae859ae715742b48c197b304d9f7fc6)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:05.122341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:05.128475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12a9e3-9ffb-4917-b86f-fb8a91139227 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:05.132154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12a9e3-9ffb-4917-b86f-fb8a91139227 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:05.132154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12a9e3-9ffb-4917-b86f-fb8a91139227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:05.177964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-6d12a9e3-9ffb-4917-b86f-fb8a91139227 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:05.182743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6d12a9e3-9ffb-4917-b86f-fb8a91139227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:05.198188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1060/2124] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:05.216671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12a9e3-9ffb-4917-b86f-fb8a91139227 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:05.227249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12a9e3-9ffb-4917-b86f-fb8a91139227 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:05.233804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59233154-f8db-4ef1-8c5e-2463a8c30744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:05.234464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59233154-f8db-4ef1-8c5e-2463a8c30744 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:05.234752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59233154-f8db-4ef1-8c5e-2463a8c30744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:05.235028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59233154-f8db-4ef1-8c5e-2463a8c30744 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:05.235863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59233154-f8db-4ef1-8c5e-2463a8c30744 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:53:05.236174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59233154-f8db-4ef1-8c5e-2463a8c30744 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:05.237824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59233154-f8db-4ef1-8c5e-2463a8c30744 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:05.251799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1061/2125] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:53:05.263536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59233154-f8db-4ef1-8c5e-2463a8c30744 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:05.281136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59233154-f8db-4ef1-8c5e-2463a8c30744 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:05.282642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce88f08-70bc-4601-bb88-0f2fa56255ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:05.282642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce88f08-70bc-4601-bb88-0f2fa56255ff admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:05.282642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce88f08-70bc-4601-bb88-0f2fa56255ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:05.282642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce88f08-70bc-4601-bb88-0f2fa56255ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:05.284283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce88f08-70bc-4601-bb88-0f2fa56255ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:05.285188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce88f08-70bc-4601-bb88-0f2fa56255ff admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:05.307986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:05.318219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae859ae715742b48c197b304d9f7fc6', 'request_id': 'req-ae66945b-bfad-41db-ab0f-61dcf833880c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:05.327300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae859ae715742b48c197b304d9f7fc6', 'request_id': 'req-ae66945b-bfad-41db-ab0f-61dcf833880c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:05.440617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1062/2126] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:05 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:05.444037 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce88f08-70bc-4601-bb88-0f2fa56255ff admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:05.479107 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae859ae715742b48c197b304d9f7fc6', 'request_id': 'req-ae66945b-bfad-41db-ab0f-61dcf833880c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:05.496841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] The token cache is being invalidate because project fae859ae715742b48c197b304d9f7fc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:05.496841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:05.496841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:05.496841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1065/2127] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:04 2022] DELETE /identity/v3/projects/fae859ae715742b48c197b304d9f7fc6 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:05.504555 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:05.506213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:05.543797 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce88f08-70bc-4601-bb88-0f2fa56255ff admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:05.549780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9917b4-ce13-457b-bf42-ce51079a307e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:05.550493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9917b4-ce13-457b-bf42-ce51079a307e admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:05.550772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9917b4-ce13-457b-bf42-ce51079a307e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:05.551658 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9917b4-ce13-457b-bf42-ce51079a307e admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:05.551658 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9917b4-ce13-457b-bf42-ce51079a307e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:05.553867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9917b4-ce13-457b-bf42-ce51079a307e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:05.581720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66945b-bfad-41db-ab0f-61dcf833880c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:05.587455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:05.588720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:05.588720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:05.588974 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] PATH_INFO: `/v3/users/62073fc54ec8489b839d52d03a1f918c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:05.590188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62073fc54ec8489b839d52d03a1f918c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:05.592637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:05.665365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1063/2128] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:05 2022] POST /identity/v3/users => generated 291 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:53:05.672212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9917b4-ce13-457b-bf42-ce51079a307e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:05.674202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bd9917b4-ce13-457b-bf42-ce51079a307e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:05.687762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9917b4-ce13-457b-bf42-ce51079a307e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:05.692900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac4d1b-7d51-4098-920e-4febcec05b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:05.693441 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac4d1b-7d51-4098-920e-4febcec05b33 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:05.693718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac4d1b-7d51-4098-920e-4febcec05b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:05.693988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac4d1b-7d51-4098-920e-4febcec05b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:05.694757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac4d1b-7d51-4098-920e-4febcec05b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:05.696362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac4d1b-7d51-4098-920e-4febcec05b33 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:05.710461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1064/2129] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:53:05.715639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac4d1b-7d51-4098-920e-4febcec05b33 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:05.737060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac4d1b-7d51-4098-920e-4febcec05b33 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:05.742497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd6c5b-c0df-4ffd-a5e9-ac8d5672bfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:05.743079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd6c5b-c0df-4ffd-a5e9-ac8d5672bfad admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:05.743519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd6c5b-c0df-4ffd-a5e9-ac8d5672bfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:05.743892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd6c5b-c0df-4ffd-a5e9-ac8d5672bfad admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:05.746471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd6c5b-c0df-4ffd-a5e9-ac8d5672bfad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:05.747008 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd6c5b-c0df-4ffd-a5e9-ac8d5672bfad admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:05.800398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1065/2130] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:05.801775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd6c5b-c0df-4ffd-a5e9-ac8d5672bfad admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:05.813421 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd6c5b-c0df-4ffd-a5e9-ac8d5672bfad admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:05.819126 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468d67e-81f2-4bff-9911-3f2642b3f6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:05.819949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468d67e-81f2-4bff-9911-3f2642b3f6a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:05.820301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468d67e-81f2-4bff-9911-3f2642b3f6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:05.820650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468d67e-81f2-4bff-9911-3f2642b3f6a4 admin admin] PATH_INFO: `/v3/projects/7d3ce3d479a646d0be9d9c61d8b27123/users/af1a77c0c4b64f27be060b5b32a52c3a/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:05.844232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468d67e-81f2-4bff-9911-3f2642b3f6a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d3ce3d479a646d0be9d9c61d8b27123, user_id=af1a77c0c4b64f27be060b5b32a52c3a, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:05.850342 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468d67e-81f2-4bff-9911-3f2642b3f6a4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:05.965377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62073fc54ec8489b839d52d03a1f918c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:06.007775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62073fc54ec8489b839d52d03a1f918c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:06.019092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62073fc54ec8489b839d52d03a1f918c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:06.035640 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1066/2131] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:05 2022] PUT /identity/v3/projects/7d3ce3d479a646d0be9d9c61d8b27123/users/af1a77c0c4b64f27be060b5b32a52c3a/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:06.052105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcf01d-89dc-45bb-b7e9-0ec2784ff1e9 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:06.052105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcf01d-89dc-45bb-b7e9-0ec2784ff1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:06.052105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcf01d-89dc-45bb-b7e9-0ec2784ff1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:06.132058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-e8bcf01d-89dc-45bb-b7e9-0ec2784ff1e9 None None] MFA Rules not processed for user `af1a77c0c4b64f27be060b5b32a52c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:06.135408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e8bcf01d-89dc-45bb-b7e9-0ec2784ff1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:06.214188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62073fc54ec8489b839d52d03a1f918c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:06.214824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62073fc54ec8489b839d52d03a1f918c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:06.225075 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1066/2132] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:05 2022] DELETE /identity/v3/users/62073fc54ec8489b839d52d03a1f918c => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:06.236408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1067/2133] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:06.259342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:06.263542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:06.316641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:06.398651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e56e3-11f7-4f40-a814-0be135829df2 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:06.404262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf53fa0-f335-4525-a12c-3c193d89fc8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:06.405099 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf53fa0-f335-4525-a12c-3c193d89fc8c service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:06.406043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf53fa0-f335-4525-a12c-3c193d89fc8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:06.406043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf53fa0-f335-4525-a12c-3c193d89fc8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:06.408301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fbf53fa0-f335-4525-a12c-3c193d89fc8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:06.520854 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf53fa0-f335-4525-a12c-3c193d89fc8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:06.523750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf53fa0-f335-4525-a12c-3c193d89fc8c service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:06.541426 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1067/2134] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:06.547588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:06.595048 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:06.957074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcf01d-89dc-45bb-b7e9-0ec2784ff1e9 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:07.074840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcf01d-89dc-45bb-b7e9-0ec2784ff1e9 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:07.217861 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:07.218681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:07.219015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:07.219362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] PATH_INFO: `/v3/projects/89b235e76ca44aa78ddfb001e654c95b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:07.220804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b235e76ca44aa78ddfb001e654c95b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:07.223221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:07.271053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6e2f8-49c1-4b96-8ec1-66c289b10d34 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:07.271053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6e2f8-49c1-4b96-8ec1-66c289b10d34 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:07.271053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6e2f8-49c1-4b96-8ec1-66c289b10d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:07.298518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-40d6e2f8-49c1-4b96-8ec1-66c289b10d34 None None] MFA Rules not processed for user `74d57012e9a04960b6c5dde2243dafbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:07.300872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-40d6e2f8-49c1-4b96-8ec1-66c289b10d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:07.380430 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b235e76ca44aa78ddfb001e654c95b', 'request_id': 'req-8c239f75-e81c-4214-be19-7fe784df2b6d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:07.386313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1068/2135] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:53:07.460383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b235e76ca44aa78ddfb001e654c95b', 'request_id': 'req-8c239f75-e81c-4214-be19-7fe784df2b6d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:07.473364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b235e76ca44aa78ddfb001e654c95b', 'request_id': 'req-8c239f75-e81c-4214-be19-7fe784df2b6d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:07.488911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] The token cache is being invalidate because project 89b235e76ca44aa78ddfb001e654c95b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:07.489183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:07.490409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8c239f75-e81c-4214-be19-7fe784df2b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:07.491816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1068/2136] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:06 2022] DELETE /identity/v3/projects/89b235e76ca44aa78ddfb001e654c95b => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:07.498132 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d1138-5ea5-4c4c-a827-9091ec20f857 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:07.500396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d1138-5ea5-4c4c-a827-9091ec20f857 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:07.500927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d1138-5ea5-4c4c-a827-9091ec20f857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:07.527632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d90d1138-5ea5-4c4c-a827-9091ec20f857 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:07.529742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d90d1138-5ea5-4c4c-a827-9091ec20f857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:07.688679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1069/2137] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:07.696943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22378e9a-63bc-4158-9655-bd7923e62027 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:07.697974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22378e9a-63bc-4158-9655-bd7923e62027 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:07.697974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22378e9a-63bc-4158-9655-bd7923e62027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:07.727810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-22378e9a-63bc-4158-9655-bd7923e62027 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:07.730598 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-22378e9a-63bc-4158-9655-bd7923e62027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:07.749972 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1069/2138] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:07.756287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d1138-5ea5-4c4c-a827-9091ec20f857 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:07.776083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d1138-5ea5-4c4c-a827-9091ec20f857 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:07.785413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dda93-4fdb-4f38-ae64-6429d39d5e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:07.786403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dda93-4fdb-4f38-ae64-6429d39d5e95 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:07.786904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dda93-4fdb-4f38-ae64-6429d39d5e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:07.787446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dda93-4fdb-4f38-ae64-6429d39d5e95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:07.788642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93dda93-4fdb-4f38-ae64-6429d39d5e95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:53:07.789206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93dda93-4fdb-4f38-ae64-6429d39d5e95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:07.792536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93dda93-4fdb-4f38-ae64-6429d39d5e95 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:07.816247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1070/2139] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:53:07.823271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22378e9a-63bc-4158-9655-bd7923e62027 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:07.830928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:07.836833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22378e9a-63bc-4158-9655-bd7923e62027 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:07.843868 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab263c-94a5-4752-ac15-d5850aac2fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:07.844628 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab263c-94a5-4752-ac15-d5850aac2fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:07.845578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab263c-94a5-4752-ac15-d5850aac2fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:07.845578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab263c-94a5-4752-ac15-d5850aac2fe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:07.846029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab263c-94a5-4752-ac15-d5850aac2fe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:07.848585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab263c-94a5-4752-ac15-d5850aac2fe8 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:07.949772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1070/2140] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:07 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:07.956136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dda93-4fdb-4f38-ae64-6429d39d5e95 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:07.969702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dda93-4fdb-4f38-ae64-6429d39d5e95 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:08.025959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:08.067090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beba0f9-06eb-493a-81fb-e402e8efc524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:08.068043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beba0f9-06eb-493a-81fb-e402e8efc524 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:08.068514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beba0f9-06eb-493a-81fb-e402e8efc524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:08.068937 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beba0f9-06eb-493a-81fb-e402e8efc524 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:08.069941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beba0f9-06eb-493a-81fb-e402e8efc524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:08.072436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beba0f9-06eb-493a-81fb-e402e8efc524 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:08.222474 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1071/2141] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:07 2022] POST /identity/v3/users => generated 284 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:53:08.227119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab263c-94a5-4752-ac15-d5850aac2fe8 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:08.239660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab263c-94a5-4752-ac15-d5850aac2fe8 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:08.245267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20fb3f-eb93-476b-97d2-c47865916bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:08.245734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20fb3f-eb93-476b-97d2-c47865916bfb admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:08.246040 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20fb3f-eb93-476b-97d2-c47865916bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:08.246394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20fb3f-eb93-476b-97d2-c47865916bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:08.247116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20fb3f-eb93-476b-97d2-c47865916bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:08.249019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20fb3f-eb93-476b-97d2-c47865916bfb admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:08.261494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1071/2142] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:08.266170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beba0f9-06eb-493a-81fb-e402e8efc524 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:08.285804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beba0f9-06eb-493a-81fb-e402e8efc524 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:08.291285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292065b-15ac-43e4-9894-e2331bb60edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:08.292098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292065b-15ac-43e4-9894-e2331bb60edd admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:08.292418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292065b-15ac-43e4-9894-e2331bb60edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:08.292736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292065b-15ac-43e4-9894-e2331bb60edd admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:08.293490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292065b-15ac-43e4-9894-e2331bb60edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:08.295861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292065b-15ac-43e4-9894-e2331bb60edd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:08.310898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1072/2143] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:08.320506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20fb3f-eb93-476b-97d2-c47865916bfb admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:08.340009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20fb3f-eb93-476b-97d2-c47865916bfb admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:08.361739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d737cc-befd-4bf0-a910-7843fabdbdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:08.362859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d737cc-befd-4bf0-a910-7843fabdbdb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:08.363335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d737cc-befd-4bf0-a910-7843fabdbdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:08.363840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d737cc-befd-4bf0-a910-7843fabdbdb4 admin admin] PATH_INFO: `/v3/projects/d05aca5e897d41a892437e5406eaf803/users/d265c036665f42a3863224a4c823d7f0/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:08.411704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d737cc-befd-4bf0-a910-7843fabdbdb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05aca5e897d41a892437e5406eaf803, user_id=d265c036665f42a3863224a4c823d7f0, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:08.416374 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d737cc-befd-4bf0-a910-7843fabdbdb4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:08.546783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1072/2144] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:08 2022] PUT /identity/v3/projects/d05aca5e897d41a892437e5406eaf803/users/d265c036665f42a3863224a4c823d7f0/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:08.549526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b98ac-2548-4ce7-a1df-098072bb40de None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:08.550377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b98ac-2548-4ce7-a1df-098072bb40de None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:08.550377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b98ac-2548-4ce7-a1df-098072bb40de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:08.598281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b69b98ac-2548-4ce7-a1df-098072bb40de None None] MFA Rules not processed for user `d265c036665f42a3863224a4c823d7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:08.600232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b69b98ac-2548-4ce7-a1df-098072bb40de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:08.684593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d737cc-befd-4bf0-a910-7843fabdbdb4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:08.695794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-09d737cc-befd-4bf0-a910-7843fabdbdb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:08.707428 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1073/2145] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:53:08.713779 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b98ac-2548-4ce7-a1df-098072bb40de None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:08.831767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b98ac-2548-4ce7-a1df-098072bb40de None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:08.839457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059255a-1b20-4796-9899-46f654154233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:08.840502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059255a-1b20-4796-9899-46f654154233 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:08.842405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059255a-1b20-4796-9899-46f654154233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:08.842962 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059255a-1b20-4796-9899-46f654154233 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:08.843674 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059255a-1b20-4796-9899-46f654154233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:08.845670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059255a-1b20-4796-9899-46f654154233 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:08.868902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d737cc-befd-4bf0-a910-7843fabdbdb4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:08.878250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e8883-6460-41aa-a6c4-5aa99e05fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:08.879389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e8883-6460-41aa-a6c4-5aa99e05fb03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:08.879902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e8883-6460-41aa-a6c4-5aa99e05fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:08.880494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e8883-6460-41aa-a6c4-5aa99e05fb03 admin admin] PATH_INFO: `/v3/users/ca316d3889124ac2a16be6b86326d020` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:08.883682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e8883-6460-41aa-a6c4-5aa99e05fb03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca316d3889124ac2a16be6b86326d020)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:08.886462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e8883-6460-41aa-a6c4-5aa99e05fb03 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:08.959379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1074/2146] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:08 2022] POST /identity/v3/projects => generated 369 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:53:08.963525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059255a-1b20-4796-9899-46f654154233 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:08.990357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:09.030552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059255a-1b20-4796-9899-46f654154233 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:09.036424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea631c1-47fa-428a-ba46-dd5069a32b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:09.037021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea631c1-47fa-428a-ba46-dd5069a32b6e admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:09.037298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea631c1-47fa-428a-ba46-dd5069a32b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:09.037572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea631c1-47fa-428a-ba46-dd5069a32b6e admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:09.038209 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea631c1-47fa-428a-ba46-dd5069a32b6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:09.040227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea631c1-47fa-428a-ba46-dd5069a32b6e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:09.138854 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1075/2147] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:08 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:53:09.146465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea631c1-47fa-428a-ba46-dd5069a32b6e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:09.163069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea631c1-47fa-428a-ba46-dd5069a32b6e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:09.163479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b2e8883-6460-41aa-a6c4-5aa99e05fb03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca316d3889124ac2a16be6b86326d020'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:09.176481 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33a0bc-21a2-4689-b0e8-977d08c35c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:09.176481 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33a0bc-21a2-4689-b0e8-977d08c35c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:09.176481 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33a0bc-21a2-4689-b0e8-977d08c35c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:09.176481 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33a0bc-21a2-4689-b0e8-977d08c35c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:09.177227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33a0bc-21a2-4689-b0e8-977d08c35c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:09.181293 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33a0bc-21a2-4689-b0e8-977d08c35c36 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:09.197925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1076/2148] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:09.204297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33a0bc-21a2-4689-b0e8-977d08c35c36 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:09.218123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33a0bc-21a2-4689-b0e8-977d08c35c36 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:09.219615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b2e8883-6460-41aa-a6c4-5aa99e05fb03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca316d3889124ac2a16be6b86326d020'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:09.224830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf95b53-de7b-4a2a-ac40-f74128cbad88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:09.225700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf95b53-de7b-4a2a-ac40-f74128cbad88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:09.225977 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf95b53-de7b-4a2a-ac40-f74128cbad88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:09.226278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf95b53-de7b-4a2a-ac40-f74128cbad88 admin admin] PATH_INFO: `/v3/projects/dc7699e93730481f837fef4aa3f356c3/users/12042e94d08143ca88b0083d53484098/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:09.235588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b2e8883-6460-41aa-a6c4-5aa99e05fb03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca316d3889124ac2a16be6b86326d020'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:09.253298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf95b53-de7b-4a2a-ac40-f74128cbad88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7699e93730481f837fef4aa3f356c3, user_id=12042e94d08143ca88b0083d53484098, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:09.256242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf95b53-de7b-4a2a-ac40-f74128cbad88 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:09.330576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b2e8883-6460-41aa-a6c4-5aa99e05fb03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca316d3889124ac2a16be6b86326d020'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:09.331706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5b2e8883-6460-41aa-a6c4-5aa99e05fb03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca316d3889124ac2a16be6b86326d020'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:09.353943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1073/2149] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:08 2022] DELETE /identity/v3/users/ca316d3889124ac2a16be6b86326d020 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:09.361674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20ef54-cc04-444c-9419-21f3502054e8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:09.362185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20ef54-cc04-444c-9419-21f3502054e8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:09.362717 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20ef54-cc04-444c-9419-21f3502054e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:09.394187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1077/2150] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:09 2022] PUT /identity/v3/projects/dc7699e93730481f837fef4aa3f356c3/users/12042e94d08143ca88b0083d53484098/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:09.396553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-ab20ef54-cc04-444c-9419-21f3502054e8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:09.398615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf95b53-de7b-4a2a-ac40-f74128cbad88 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:09.398860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ab20ef54-cc04-444c-9419-21f3502054e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:09.400840 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5cf95b53-de7b-4a2a-ac40-f74128cbad88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:09.449201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:09.509663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf95b53-de7b-4a2a-ac40-f74128cbad88 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:09.523455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81c882-5fa4-4d0e-9e1b-45add9442153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:09.524505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81c882-5fa4-4d0e-9e1b-45add9442153 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:09.525046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81c882-5fa4-4d0e-9e1b-45add9442153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:09.525532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81c882-5fa4-4d0e-9e1b-45add9442153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:09.527436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81c882-5fa4-4d0e-9e1b-45add9442153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:09.532022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81c882-5fa4-4d0e-9e1b-45add9442153 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:09.559307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1078/2151] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:53:09.560744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1074/2152] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:09.561777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81c882-5fa4-4d0e-9e1b-45add9442153 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:09.568017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20ef54-cc04-444c-9419-21f3502054e8 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:09.573283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ab20ef54-cc04-444c-9419-21f3502054e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:09.580872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81c882-5fa4-4d0e-9e1b-45add9442153 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:09.591141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91c1ea-b5c4-4462-80f7-4991f329ff7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:09.592665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91c1ea-b5c4-4462-80f7-4991f329ff7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:09.593349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91c1ea-b5c4-4462-80f7-4991f329ff7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:09.593914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91c1ea-b5c4-4462-80f7-4991f329ff7a admin admin] PATH_INFO: `/v3/projects/dc7699e93730481f837fef4aa3f356c3/users/12042e94d08143ca88b0083d53484098/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:09.600628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91c1ea-b5c4-4462-80f7-4991f329ff7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7699e93730481f837fef4aa3f356c3, user_id=12042e94d08143ca88b0083d53484098, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:09.604886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91c1ea-b5c4-4462-80f7-4991f329ff7a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:09.644154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1079/2153] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:09 2022] PUT /identity/v3/projects/dc7699e93730481f837fef4aa3f356c3/users/12042e94d08143ca88b0083d53484098/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:09.652106 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acd107-08e7-4fef-9562-050096fb716f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:09.652392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acd107-08e7-4fef-9562-050096fb716f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:09.652861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acd107-08e7-4fef-9562-050096fb716f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:09.692288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-20acd107-08e7-4fef-9562-050096fb716f None None] MFA Rules not processed for user `12042e94d08143ca88b0083d53484098`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:09.693960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-20acd107-08e7-4fef-9562-050096fb716f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:09.739517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20ef54-cc04-444c-9419-21f3502054e8 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:09.772198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:09.826587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1080/2154] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:09.838717 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20acd107-08e7-4fef-9562-050096fb716f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:09.839015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964094a-f213-4c9d-aa15-35d2d7e79f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:09.839917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964094a-f213-4c9d-aa15-35d2d7e79f56 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:09.840186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-20acd107-08e7-4fef-9562-050096fb716f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:09.840489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964094a-f213-4c9d-aa15-35d2d7e79f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:09.841748 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964094a-f213-4c9d-aa15-35d2d7e79f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:09.949806 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20acd107-08e7-4fef-9562-050096fb716f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:09.955264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964094a-f213-4c9d-aa15-35d2d7e79f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:09.956499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a1f38-bccf-4cbf-8f0a-356a27866c8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:09.957133 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a1f38-bccf-4cbf-8f0a-356a27866c8a service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:09.957412 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a1f38-bccf-4cbf-8f0a-356a27866c8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:09.957684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a1f38-bccf-4cbf-8f0a-356a27866c8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:09.959201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964094a-f213-4c9d-aa15-35d2d7e79f56 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:09.969592 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a1f38-bccf-4cbf-8f0a-356a27866c8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:09.971799 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a1f38-bccf-4cbf-8f0a-356a27866c8a service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:09.979723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1075/2155] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:09.990872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1081/2156] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:53:10.349540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964094a-f213-4c9d-aa15-35d2d7e79f56 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:10.351681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b964094a-f213-4c9d-aa15-35d2d7e79f56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:10.369466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964094a-f213-4c9d-aa15-35d2d7e79f56 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:10.376148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f35aa-c9b7-4de3-93f1-e694d140e649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:10.377113 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f35aa-c9b7-4de3-93f1-e694d140e649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:10.377113 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f35aa-c9b7-4de3-93f1-e694d140e649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:10.377340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f35aa-c9b7-4de3-93f1-e694d140e649 admin admin] PATH_INFO: `/v3/users/af1a77c0c4b64f27be060b5b32a52c3a` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:10.378934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f35aa-c9b7-4de3-93f1-e694d140e649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1a77c0c4b64f27be060b5b32a52c3a)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:10.380951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f35aa-c9b7-4de3-93f1-e694d140e649 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:10.536033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:10.666681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a1f38-bccf-4cbf-8f0a-356a27866c8a service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:10.670150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e27a1f38-bccf-4cbf-8f0a-356a27866c8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:10.705742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a1f38-bccf-4cbf-8f0a-356a27866c8a service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:10.724843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:10.733428 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3a4f35aa-c9b7-4de3-93f1-e694d140e649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1a77c0c4b64f27be060b5b32a52c3a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:10.736880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:10.737240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:10.739094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] PATH_INFO: `/v3/projects/fca828c8bda9425a915a390768976f99` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:10.742498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca828c8bda9425a915a390768976f99)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:10.749622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:10.789692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3a4f35aa-c9b7-4de3-93f1-e694d140e649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1a77c0c4b64f27be060b5b32a52c3a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:10.817754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3a4f35aa-c9b7-4de3-93f1-e694d140e649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1a77c0c4b64f27be060b5b32a52c3a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:10.902647 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca828c8bda9425a915a390768976f99', 'request_id': 'req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:10.912740 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca828c8bda9425a915a390768976f99', 'request_id': 'req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:10.930482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3a4f35aa-c9b7-4de3-93f1-e694d140e649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1a77c0c4b64f27be060b5b32a52c3a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:10.931307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3a4f35aa-c9b7-4de3-93f1-e694d140e649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1a77c0c4b64f27be060b5b32a52c3a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:10.965482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1076/2157] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:10 2022] DELETE /identity/v3/users/af1a77c0c4b64f27be060b5b32a52c3a => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:10.965482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950cf1c-e44a-4262-9548-9a95f7c3a6cc None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:10.965482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950cf1c-e44a-4262-9548-9a95f7c3a6cc None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:10.965482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950cf1c-e44a-4262-9548-9a95f7c3a6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:10.992451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-8950cf1c-e44a-4262-9548-9a95f7c3a6cc None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:10.996289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8950cf1c-e44a-4262-9548-9a95f7c3a6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:11.014296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca828c8bda9425a915a390768976f99', 'request_id': 'req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:11.036627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] The token cache is being invalidate because project fca828c8bda9425a915a390768976f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:11.036967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:11.039115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-106577bc-f7d2-4b3a-bd82-4d6ff72cd744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:11.045662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1082/2158] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:10 2022] DELETE /identity/v3/projects/fca828c8bda9425a915a390768976f99 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:11.062465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374c8eb-ad3f-49f2-a56d-807fba12a2bb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:11.062465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374c8eb-ad3f-49f2-a56d-807fba12a2bb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:11.062465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374c8eb-ad3f-49f2-a56d-807fba12a2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:11.076227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2374c8eb-ad3f-49f2-a56d-807fba12a2bb None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:11.077747 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2374c8eb-ad3f-49f2-a56d-807fba12a2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:11.094006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1077/2159] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:11.134172 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950cf1c-e44a-4262-9548-9a95f7c3a6cc None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:11.136291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8950cf1c-e44a-4262-9548-9a95f7c3a6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:11.159212 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1083/2160] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:11.168724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7b88-cf31-4e92-a414-ae181153bd7c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:11.168988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7b88-cf31-4e92-a414-ae181153bd7c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:11.169364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7b88-cf31-4e92-a414-ae181153bd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:11.197142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ba5c7b88-cf31-4e92-a414-ae181153bd7c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:11.199056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ba5c7b88-cf31-4e92-a414-ae181153bd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:11.219239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1084/2161] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:11.226810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7b88-cf31-4e92-a414-ae181153bd7c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:11.237008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:11.243343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7b88-cf31-4e92-a414-ae181153bd7c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:11.251667 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195617b-5ead-448d-8f28-c506a91a75b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:11.257542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195617b-5ead-448d-8f28-c506a91a75b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:11.257542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195617b-5ead-448d-8f28-c506a91a75b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:11.257542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195617b-5ead-448d-8f28-c506a91a75b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:11.257542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195617b-5ead-448d-8f28-c506a91a75b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:53:11.257542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195617b-5ead-448d-8f28-c506a91a75b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:11.257542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195617b-5ead-448d-8f28-c506a91a75b9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:11.263542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950cf1c-e44a-4262-9548-9a95f7c3a6cc None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:11.268891 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac87e85-6951-46c9-9a9c-0954987e05f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:11.269524 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac87e85-6951-46c9-9a9c-0954987e05f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:11.269808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac87e85-6951-46c9-9a9c-0954987e05f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:11.270113 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac87e85-6951-46c9-9a9c-0954987e05f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:11.273574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1085/2162] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:53:11.280197 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195617b-5ead-448d-8f28-c506a91a75b9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:11.292019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac87e85-6951-46c9-9a9c-0954987e05f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:11.294057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac87e85-6951-46c9-9a9c-0954987e05f0 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:11.294700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195617b-5ead-448d-8f28-c506a91a75b9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:11.301893 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a488be-b16e-43c1-a3ae-29010de52b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:11.302552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a488be-b16e-43c1-a3ae-29010de52b14 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:11.302865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a488be-b16e-43c1-a3ae-29010de52b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:11.303334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a488be-b16e-43c1-a3ae-29010de52b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:11.303946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a488be-b16e-43c1-a3ae-29010de52b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:11.305935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a488be-b16e-43c1-a3ae-29010de52b14 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:11.312208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1078/2163] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:53:11.451435 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1086/2164] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:11 2022] POST /identity/v3/projects => generated 389 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:53:11.452224 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac87e85-6951-46c9-9a9c-0954987e05f0 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:11.476425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac87e85-6951-46c9-9a9c-0954987e05f0 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:11.577823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:11.595639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e0fc9-fe95-4bf0-98ce-64f0966656f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:11.596540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e0fc9-fe95-4bf0-98ce-64f0966656f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:11.596826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e0fc9-fe95-4bf0-98ce-64f0966656f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:11.597105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e0fc9-fe95-4bf0-98ce-64f0966656f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:11.597767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e0fc9-fe95-4bf0-98ce-64f0966656f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:11.599881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e0fc9-fe95-4bf0-98ce-64f0966656f3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:11.753632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a488be-b16e-43c1-a3ae-29010de52b14 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:11.755024 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1079/2165] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:11 2022] POST /identity/v3/users => generated 295 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:11.767361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a488be-b16e-43c1-a3ae-29010de52b14 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:11.780221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75a017-76d5-4c9a-aa21-8df265586416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:11.781055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a017-76d5-4c9a-aa21-8df265586416 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:11.781255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a017-76d5-4c9a-aa21-8df265586416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:11.781560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a017-76d5-4c9a-aa21-8df265586416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:11.782348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75a017-76d5-4c9a-aa21-8df265586416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:11.784287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75a017-76d5-4c9a-aa21-8df265586416 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:11.799189 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1087/2166] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:11.803807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e0fc9-fe95-4bf0-98ce-64f0966656f3 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:11.815630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e0fc9-fe95-4bf0-98ce-64f0966656f3 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:11.821097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbb8f7-0507-43b1-acd4-e57ef47934e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:11.821723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbb8f7-0507-43b1-acd4-e57ef47934e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:11.822048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbb8f7-0507-43b1-acd4-e57ef47934e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:11.822278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbb8f7-0507-43b1-acd4-e57ef47934e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:11.823084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbb8f7-0507-43b1-acd4-e57ef47934e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:11.824950 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbb8f7-0507-43b1-acd4-e57ef47934e5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:11.838530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1080/2167] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:53:11.843794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75a017-76d5-4c9a-aa21-8df265586416 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:11.855347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75a017-76d5-4c9a-aa21-8df265586416 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:11.861819 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b75fd-446a-4cf1-a4b2-d18b08d2a39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:11.862693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b75fd-446a-4cf1-a4b2-d18b08d2a39c admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:11.862981 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b75fd-446a-4cf1-a4b2-d18b08d2a39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:11.863260 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b75fd-446a-4cf1-a4b2-d18b08d2a39c admin admin] PATH_INFO: `/v3/projects/f88335e0c0ea4d7eb0ab447fb25f5178/users/cdb3bd9006c7497a821e4e5226f8ac01/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:11.883482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b75fd-446a-4cf1-a4b2-d18b08d2a39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f88335e0c0ea4d7eb0ab447fb25f5178, user_id=cdb3bd9006c7497a821e4e5226f8ac01, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:11.888274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b75fd-446a-4cf1-a4b2-d18b08d2a39c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:12.044975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1088/2168] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:11 2022] PUT /identity/v3/projects/f88335e0c0ea4d7eb0ab447fb25f5178/users/cdb3bd9006c7497a821e4e5226f8ac01/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:12.057769 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f912f-c4f6-4c3a-848e-3622f57a5875 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:12.058606 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f912f-c4f6-4c3a-848e-3622f57a5875 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:12.059018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f912f-c4f6-4c3a-848e-3622f57a5875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:12.088625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b75fd-446a-4cf1-a4b2-d18b08d2a39c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:12.088625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-722b75fd-446a-4cf1-a4b2-d18b08d2a39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:12.154061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-ec5f912f-c4f6-4c3a-848e-3622f57a5875 None None] MFA Rules not processed for user `cdb3bd9006c7497a821e4e5226f8ac01`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:12.159217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ec5f912f-c4f6-4c3a-848e-3622f57a5875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:12.264170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1081/2169] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:12.278011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f912f-c4f6-4c3a-848e-3622f57a5875 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:12.280842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b75fd-446a-4cf1-a4b2-d18b08d2a39c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:12.283004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ec5f912f-c4f6-4c3a-848e-3622f57a5875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:12.296058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:12.296831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:12.310024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:12.311490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] PATH_INFO: `/v3/projects/7d3ce3d479a646d0be9d9c61d8b27123` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:12.315730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d3ce3d479a646d0be9d9c61d8b27123)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:12.320521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:12.409113 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f912f-c4f6-4c3a-848e-3622f57a5875 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:12.413241 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa725e-806e-4814-b24b-c3392dc88dea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:12.413900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa725e-806e-4814-b24b-c3392dc88dea service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:12.415611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa725e-806e-4814-b24b-c3392dc88dea service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:12.415611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa725e-806e-4814-b24b-c3392dc88dea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:12.425628 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa725e-806e-4814-b24b-c3392dc88dea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:12.427330 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa725e-806e-4814-b24b-c3392dc88dea service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:12.452006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1082/2170] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:12.490153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d3ce3d479a646d0be9d9c61d8b27123', 'request_id': 'req-5086f455-f2f6-46c6-933c-9c87e44b5668'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:12.525492 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d3ce3d479a646d0be9d9c61d8b27123', 'request_id': 'req-5086f455-f2f6-46c6-933c-9c87e44b5668'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:12.709883 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d3ce3d479a646d0be9d9c61d8b27123', 'request_id': 'req-5086f455-f2f6-46c6-933c-9c87e44b5668'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:12.717291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] The token cache is being invalidate because project 7d3ce3d479a646d0be9d9c61d8b27123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:12.717575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:12.718257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5086f455-f2f6-46c6-933c-9c87e44b5668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:12.720004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1089/2171] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:12 2022] DELETE /identity/v3/projects/7d3ce3d479a646d0be9d9c61d8b27123 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:12.725548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5c908-c78a-4729-98f4-faeff376493b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:12.726081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5c908-c78a-4729-98f4-faeff376493b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:12.726505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5c908-c78a-4729-98f4-faeff376493b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:12.750693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-cee5c908-c78a-4729-98f4-faeff376493b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:12.751904 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-cee5c908-c78a-4729-98f4-faeff376493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:12.846412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1083/2172] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:12.851366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2632cea-ea38-46d5-8eb5-7dc9c3546719 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:12.851366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2632cea-ea38-46d5-8eb5-7dc9c3546719 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:12.851366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2632cea-ea38-46d5-8eb5-7dc9c3546719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:12.852790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:12.892818 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f2632cea-ea38-46d5-8eb5-7dc9c3546719 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:12.894079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f2632cea-ea38-46d5-8eb5-7dc9c3546719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:12.907548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1090/2173] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:12.915053 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5c908-c78a-4729-98f4-faeff376493b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:12.933887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5c908-c78a-4729-98f4-faeff376493b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:12.943814 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5411e-7d32-4bb2-b021-d5ff02bdb80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:12.944539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5411e-7d32-4bb2-b021-d5ff02bdb80f admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:12.945148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5411e-7d32-4bb2-b021-d5ff02bdb80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:12.945576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5411e-7d32-4bb2-b021-d5ff02bdb80f admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:12.947413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5411e-7d32-4bb2-b021-d5ff02bdb80f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:53:12.947951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5411e-7d32-4bb2-b021-d5ff02bdb80f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:12.951267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5411e-7d32-4bb2-b021-d5ff02bdb80f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:12.976684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1084/2174] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:53:12.983102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2632cea-ea38-46d5-8eb5-7dc9c3546719 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:12.993391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2632cea-ea38-46d5-8eb5-7dc9c3546719 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:13.002492 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927cdf6-237e-42ad-ac1b-1cfd444ba0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:13.003347 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927cdf6-237e-42ad-ac1b-1cfd444ba0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:13.003630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927cdf6-237e-42ad-ac1b-1cfd444ba0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:13.003903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927cdf6-237e-42ad-ac1b-1cfd444ba0f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:13.004549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927cdf6-237e-42ad-ac1b-1cfd444ba0f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:13.006610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927cdf6-237e-42ad-ac1b-1cfd444ba0f5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:13.030834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:13.207159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5411e-7d32-4bb2-b021-d5ff02bdb80f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:13.209083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1091/2175] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:12 2022] POST /identity/v3/projects => generated 383 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:13.223857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5411e-7d32-4bb2-b021-d5ff02bdb80f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:13.354145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e24d37-8e4a-4181-af8e-bf9867c433f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:13.355534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e24d37-8e4a-4181-af8e-bf9867c433f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:13.356555 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e24d37-8e4a-4181-af8e-bf9867c433f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:13.357597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e24d37-8e4a-4181-af8e-bf9867c433f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:13.360548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e24d37-8e4a-4181-af8e-bf9867c433f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:13.363926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e24d37-8e4a-4181-af8e-bf9867c433f2 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:13.479675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1085/2176] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:13 2022] POST /identity/v3/users => generated 292 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:13.484326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927cdf6-237e-42ad-ac1b-1cfd444ba0f5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:13.498115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927cdf6-237e-42ad-ac1b-1cfd444ba0f5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:13.503784 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a858a80-b2c6-4510-aca5-b2ce97e05f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:13.504413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a858a80-b2c6-4510-aca5-b2ce97e05f4c admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:13.504763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a858a80-b2c6-4510-aca5-b2ce97e05f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:13.505151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a858a80-b2c6-4510-aca5-b2ce97e05f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:13.505823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a858a80-b2c6-4510-aca5-b2ce97e05f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:13.507857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a858a80-b2c6-4510-aca5-b2ce97e05f4c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:13.532107 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1092/2177] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:13.543089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e24d37-8e4a-4181-af8e-bf9867c433f2 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:13.556746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e24d37-8e4a-4181-af8e-bf9867c433f2 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:13.570664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c572a5-c864-4bf2-8e6c-6cecbadcff60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:13.570664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c572a5-c864-4bf2-8e6c-6cecbadcff60 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:13.570664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c572a5-c864-4bf2-8e6c-6cecbadcff60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:13.570664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c572a5-c864-4bf2-8e6c-6cecbadcff60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:13.571287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c572a5-c864-4bf2-8e6c-6cecbadcff60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:13.571287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c572a5-c864-4bf2-8e6c-6cecbadcff60 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:13.583404 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1086/2178] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:13.584851 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a858a80-b2c6-4510-aca5-b2ce97e05f4c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:13.602567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a858a80-b2c6-4510-aca5-b2ce97e05f4c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:13.613927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa81cb-9301-4799-9f2d-ea012af59adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:13.614982 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa81cb-9301-4799-9f2d-ea012af59adb admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:13.615408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa81cb-9301-4799-9f2d-ea012af59adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:13.615833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa81cb-9301-4799-9f2d-ea012af59adb admin admin] PATH_INFO: `/v3/projects/2f665e85ae5b4008b60765fd3076a331/users/27f78b50d76341e08e184582e8400a04/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:13.645822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa81cb-9301-4799-9f2d-ea012af59adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f665e85ae5b4008b60765fd3076a331, user_id=27f78b50d76341e08e184582e8400a04, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:13.648569 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa81cb-9301-4799-9f2d-ea012af59adb admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:13.806107 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1093/2179] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:13 2022] PUT /identity/v3/projects/2f665e85ae5b4008b60765fd3076a331/users/27f78b50d76341e08e184582e8400a04/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:13.812072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8ad0-90d0-4a9e-879d-306b30dda0be None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:13.813278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8ad0-90d0-4a9e-879d-306b30dda0be None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:13.813278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8ad0-90d0-4a9e-879d-306b30dda0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:13.852743 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-b4ad8ad0-90d0-4a9e-879d-306b30dda0be None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:13.854267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b4ad8ad0-90d0-4a9e-879d-306b30dda0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:13.970730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1087/2180] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:13.991158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa81cb-9301-4799-9f2d-ea012af59adb admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:13.994994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-91fa81cb-9301-4799-9f2d-ea012af59adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:14.101436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa81cb-9301-4799-9f2d-ea012af59adb admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:14.105945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b8272-618b-4232-b12c-51d88bf0d3e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:14.106678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b8272-618b-4232-b12c-51d88bf0d3e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:14.106956 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b8272-618b-4232-b12c-51d88bf0d3e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:14.107228 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b8272-618b-4232-b12c-51d88bf0d3e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:14.120554 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b8272-618b-4232-b12c-51d88bf0d3e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:14.122684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b8272-618b-4232-b12c-51d88bf0d3e1 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:14.134742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1094/2181] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:15.585860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad8ad0-90d0-4a9e-879d-306b30dda0be None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:15.594955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b4ad8ad0-90d0-4a9e-879d-306b30dda0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:15.648050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad8ad0-90d0-4a9e-879d-306b30dda0be None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:15.678563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:15.679678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:15.680158 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:15.680637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] PATH_INFO: `/v3/users/d265c036665f42a3863224a4c823d7f0` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:15.685589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d265c036665f42a3863224a4c823d7f0)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:15.704147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:16.160531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd265c036665f42a3863224a4c823d7f0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:16.186887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:16.190279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd265c036665f42a3863224a4c823d7f0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:16.216310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd265c036665f42a3863224a4c823d7f0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:16.314288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd265c036665f42a3863224a4c823d7f0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:16.318385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd265c036665f42a3863224a4c823d7f0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:16.349143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1781f-553f-4472-8d52-9ec39244845f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:16.350949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1781f-553f-4472-8d52-9ec39244845f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:16.351351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1781f-553f-4472-8d52-9ec39244845f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:16.353914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1088/2182] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:15 2022] DELETE /identity/v3/users/d265c036665f42a3863224a4c823d7f0 => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:16.383867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3cf1781f-553f-4472-8d52-9ec39244845f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:16.384990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3cf1781f-553f-4472-8d52-9ec39244845f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:16.540221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1095/2183] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:16.552976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:16.554820 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:16.662006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b94c5-f85d-459e-b913-d43910c3eaa2 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:16.662006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526084e5-589a-4d34-9091-22ae4582d29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:16.662006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526084e5-589a-4d34-9091-22ae4582d29d service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:16.662006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526084e5-589a-4d34-9091-22ae4582d29d service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:16.663351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526084e5-589a-4d34-9091-22ae4582d29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:16.663351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526084e5-589a-4d34-9091-22ae4582d29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:16.663351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526084e5-589a-4d34-9091-22ae4582d29d service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:16.664580 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1089/2184] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:53:16.952785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fa5de-5dc9-46b7-8d5b-e231a879702a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:16.953066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fa5de-5dc9-46b7-8d5b-e231a879702a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:16.953339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fa5de-5dc9-46b7-8d5b-e231a879702a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:16.994608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-bd7fa5de-5dc9-46b7-8d5b-e231a879702a None None] MFA Rules not processed for user `734d44438bfb4a79a68aa6705e49b1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:16.994608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bd7fa5de-5dc9-46b7-8d5b-e231a879702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:17.081299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1096/2185] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:53:17.340951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526084e5-589a-4d34-9091-22ae4582d29d service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:17.340951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-526084e5-589a-4d34-9091-22ae4582d29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:17.417044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526084e5-589a-4d34-9091-22ae4582d29d service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:17.444030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:17.445239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:17.445701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:17.446113 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] PATH_INFO: `/v3/projects/d05aca5e897d41a892437e5406eaf803` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:17.453895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05aca5e897d41a892437e5406eaf803)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:17.456225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:17.625437 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05aca5e897d41a892437e5406eaf803', 'request_id': 'req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:17.750654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05aca5e897d41a892437e5406eaf803', 'request_id': 'req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:17.760643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05aca5e897d41a892437e5406eaf803', 'request_id': 'req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:17.779992 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] The token cache is being invalidate because project d05aca5e897d41a892437e5406eaf803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:17.780508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:17.781412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:17.783320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1090/2186] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:17 2022] DELETE /identity/v3/projects/d05aca5e897d41a892437e5406eaf803 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:17.787826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fa5de-5dc9-46b7-8d5b-e231a879702a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:17.800344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fa5de-5dc9-46b7-8d5b-e231a879702a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:17.806382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:17.894725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75484e-1f2f-4c25-9e6b-569d3e9e532c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:17.895391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75484e-1f2f-4c25-9e6b-569d3e9e532c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:17.895661 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75484e-1f2f-4c25-9e6b-569d3e9e532c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:17.895937 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75484e-1f2f-4c25-9e6b-569d3e9e532c admin admin] PATH_INFO: `/v3/users/12042e94d08143ca88b0083d53484098` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:17.897476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75484e-1f2f-4c25-9e6b-569d3e9e532c admin admin] RBAC: Authorizing `identity:delete_user(user_id=12042e94d08143ca88b0083d53484098)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:17.900344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75484e-1f2f-4c25-9e6b-569d3e9e532c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:18.155533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-da75484e-1f2f-4c25-9e6b-569d3e9e532c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12042e94d08143ca88b0083d53484098'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:18.194121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-da75484e-1f2f-4c25-9e6b-569d3e9e532c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12042e94d08143ca88b0083d53484098'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:18.247769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-da75484e-1f2f-4c25-9e6b-569d3e9e532c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12042e94d08143ca88b0083d53484098'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:18.361894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-da75484e-1f2f-4c25-9e6b-569d3e9e532c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12042e94d08143ca88b0083d53484098'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:18.367968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-da75484e-1f2f-4c25-9e6b-569d3e9e532c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12042e94d08143ca88b0083d53484098'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:18.383266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1097/2187] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:17 2022] DELETE /identity/v3/users/12042e94d08143ca88b0083d53484098 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:19.306451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:19.309127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:19.367717 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc031cd-9d1b-441d-9b7f-6b8be35fc12e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:19.653933 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:19.655390 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:19.657669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:19.658628 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] PATH_INFO: `/v3/projects/dc7699e93730481f837fef4aa3f356c3` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:19.665956 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7699e93730481f837fef4aa3f356c3)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:19.671597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:19.696548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:19.978316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7699e93730481f837fef4aa3f356c3', 'request_id': 'req-b86d5705-1081-4e5b-be8f-2d20b3e18a82'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:20.054244 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7699e93730481f837fef4aa3f356c3', 'request_id': 'req-b86d5705-1081-4e5b-be8f-2d20b3e18a82'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:20.067860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7699e93730481f837fef4aa3f356c3', 'request_id': 'req-b86d5705-1081-4e5b-be8f-2d20b3e18a82'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:20.114807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] The token cache is being invalidate because project dc7699e93730481f837fef4aa3f356c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:20.117815 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:20.120620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b86d5705-1081-4e5b-be8f-2d20b3e18a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:20.128155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1091/2188] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:19 2022] DELETE /identity/v3/projects/dc7699e93730481f837fef4aa3f356c3 => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:20.135336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bca705-9713-429a-bcd7-b7ec90378c26 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:20.135983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bca705-9713-429a-bcd7-b7ec90378c26 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:20.136635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bca705-9713-429a-bcd7-b7ec90378c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:20.204051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-52bca705-9713-429a-bcd7-b7ec90378c26 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:20.206019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-52bca705-9713-429a-bcd7-b7ec90378c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:20.369767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1098/2189] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:20.380360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084e611-8dd4-49c0-b9b4-849741675e5e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:20.381332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084e611-8dd4-49c0-b9b4-849741675e5e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:20.383470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084e611-8dd4-49c0-b9b4-849741675e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:20.415241 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f084e611-8dd4-49c0-b9b4-849741675e5e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:20.416584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f084e611-8dd4-49c0-b9b4-849741675e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:20.428713 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1092/2190] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:20.437924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bca705-9713-429a-bcd7-b7ec90378c26 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:20.453925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bca705-9713-429a-bcd7-b7ec90378c26 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:20.461971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c550c2-f888-4aa6-b5b8-10350dea0964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:20.463203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c550c2-f888-4aa6-b5b8-10350dea0964 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:20.463733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c550c2-f888-4aa6-b5b8-10350dea0964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:20.464235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c550c2-f888-4aa6-b5b8-10350dea0964 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:20.465923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c550c2-f888-4aa6-b5b8-10350dea0964 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:53:20.466681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c550c2-f888-4aa6-b5b8-10350dea0964 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:20.476083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c550c2-f888-4aa6-b5b8-10350dea0964 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:20.501361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1099/2191] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:53:20.505954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084e611-8dd4-49c0-b9b4-849741675e5e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:20.517340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084e611-8dd4-49c0-b9b4-849741675e5e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:20.540065 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c45e4d-f6f7-4719-909d-b9a76581e7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:20.540065 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c45e4d-f6f7-4719-909d-b9a76581e7f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:20.540065 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c45e4d-f6f7-4719-909d-b9a76581e7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:20.540065 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c45e4d-f6f7-4719-909d-b9a76581e7f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:20.540629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c45e4d-f6f7-4719-909d-b9a76581e7f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:20.552682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c45e4d-f6f7-4719-909d-b9a76581e7f0 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:20.670859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1093/2192] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:20 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:53:20.675251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c550c2-f888-4aa6-b5b8-10350dea0964 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:20.689750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c550c2-f888-4aa6-b5b8-10350dea0964 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:20.710152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:20.794994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bda83-a7d4-46ee-9620-bbf2efb5eee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:20.795951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bda83-a7d4-46ee-9620-bbf2efb5eee0 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:20.796429 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bda83-a7d4-46ee-9620-bbf2efb5eee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:20.796921 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bda83-a7d4-46ee-9620-bbf2efb5eee0 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:20.797795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376bda83-a7d4-46ee-9620-bbf2efb5eee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:20.800447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376bda83-a7d4-46ee-9620-bbf2efb5eee0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:20.967896 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1100/2193] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:20 2022] POST /identity/v3/users => generated 293 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:53:20.973423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c45e4d-f6f7-4719-909d-b9a76581e7f0 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:21.010818 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c45e4d-f6f7-4719-909d-b9a76581e7f0 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:21.043618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231faaa4-94ab-4c27-a3c5-15eb993dd193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:21.044668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231faaa4-94ab-4c27-a3c5-15eb993dd193 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:21.045187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231faaa4-94ab-4c27-a3c5-15eb993dd193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:21.045592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231faaa4-94ab-4c27-a3c5-15eb993dd193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:21.046789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231faaa4-94ab-4c27-a3c5-15eb993dd193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:21.068778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231faaa4-94ab-4c27-a3c5-15eb993dd193 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:21.097830 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1094/2194] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:21.098628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bda83-a7d4-46ee-9620-bbf2efb5eee0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:21.115552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bda83-a7d4-46ee-9620-bbf2efb5eee0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:21.124914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b71d1-8d4b-4903-9408-a1e636d65045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:21.126969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b71d1-8d4b-4903-9408-a1e636d65045 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:21.127558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b71d1-8d4b-4903-9408-a1e636d65045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:21.128177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b71d1-8d4b-4903-9408-a1e636d65045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:21.133420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b71d1-8d4b-4903-9408-a1e636d65045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:21.135947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b71d1-8d4b-4903-9408-a1e636d65045 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:21.155498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1101/2195] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:21.160995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231faaa4-94ab-4c27-a3c5-15eb993dd193 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:21.177594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231faaa4-94ab-4c27-a3c5-15eb993dd193 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:21.188924 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab5efe-cec3-4838-82be-f2dc6ea64741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:21.192065 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab5efe-cec3-4838-82be-f2dc6ea64741 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:21.192624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab5efe-cec3-4838-82be-f2dc6ea64741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:21.193104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab5efe-cec3-4838-82be-f2dc6ea64741 admin admin] PATH_INFO: `/v3/projects/602ff95e8e44428cac7747abc1918c1d/users/7452d85fe48b4c37a5bb14cd186e7e71/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:21.251169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab5efe-cec3-4838-82be-f2dc6ea64741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=602ff95e8e44428cac7747abc1918c1d, user_id=7452d85fe48b4c37a5bb14cd186e7e71, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:21.254496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab5efe-cec3-4838-82be-f2dc6ea64741 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:21.425164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1095/2196] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:21 2022] PUT /identity/v3/projects/602ff95e8e44428cac7747abc1918c1d/users/7452d85fe48b4c37a5bb14cd186e7e71/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:21.437089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d2eaa-2c25-4ea5-a14d-9c7c32b66572 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:21.437735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d2eaa-2c25-4ea5-a14d-9c7c32b66572 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:21.438053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d2eaa-2c25-4ea5-a14d-9c7c32b66572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:21.473193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ae5d2eaa-2c25-4ea5-a14d-9c7c32b66572 None None] MFA Rules not processed for user `7452d85fe48b4c37a5bb14cd186e7e71`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:21.476010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ae5d2eaa-2c25-4ea5-a14d-9c7c32b66572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:21.584842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1102/2197] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:21.591348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab5efe-cec3-4838-82be-f2dc6ea64741 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:21.716691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab5efe-cec3-4838-82be-f2dc6ea64741 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:21.729644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f285352-a15b-47ae-9754-7235f0cf8196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:21.730362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f285352-a15b-47ae-9754-7235f0cf8196 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:21.730992 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f285352-a15b-47ae-9754-7235f0cf8196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:21.731426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f285352-a15b-47ae-9754-7235f0cf8196 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:21.733211 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f285352-a15b-47ae-9754-7235f0cf8196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:21.736736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f285352-a15b-47ae-9754-7235f0cf8196 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:21.999015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1096/2198] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:21 2022] POST /identity/v3/projects => generated 387 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:22.001014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d2eaa-2c25-4ea5-a14d-9c7c32b66572 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:22.016724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d2eaa-2c25-4ea5-a14d-9c7c32b66572 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:22.181034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21319b5d-6f44-4886-b350-b038f3f1bcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:22.182736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21319b5d-6f44-4886-b350-b038f3f1bcec admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:22.184126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21319b5d-6f44-4886-b350-b038f3f1bcec admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:22.185231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21319b5d-6f44-4886-b350-b038f3f1bcec admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:22.186953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21319b5d-6f44-4886-b350-b038f3f1bcec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:22.192658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21319b5d-6f44-4886-b350-b038f3f1bcec admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:22.354894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1103/2199] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:21 2022] POST /identity/v3/users => generated 300 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:22.362395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f285352-a15b-47ae-9754-7235f0cf8196 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:22.387146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f285352-a15b-47ae-9754-7235f0cf8196 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:22.398942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb686e0-fa0f-4cf3-87c4-10be6b905e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:22.400151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb686e0-fa0f-4cf3-87c4-10be6b905e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:22.400948 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb686e0-fa0f-4cf3-87c4-10be6b905e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:22.401552 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb686e0-fa0f-4cf3-87c4-10be6b905e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:22.403589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb686e0-fa0f-4cf3-87c4-10be6b905e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:22.406989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb686e0-fa0f-4cf3-87c4-10be6b905e6c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:22.430334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1097/2200] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:22.438736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21319b5d-6f44-4886-b350-b038f3f1bcec admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:22.456595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21319b5d-6f44-4886-b350-b038f3f1bcec admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:22.468954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8d9f8-b4b1-4edb-9226-533a3c872b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:22.470231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8d9f8-b4b1-4edb-9226-533a3c872b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:22.470923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8d9f8-b4b1-4edb-9226-533a3c872b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:22.471445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8d9f8-b4b1-4edb-9226-533a3c872b45 admin admin] PATH_INFO: `/v3/projects/b97b0316c48b49d59504a9c55fbfd0d3/users/ad388b6821e64ae0a47f01f6cb679cdb/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:22.507620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8d9f8-b4b1-4edb-9226-533a3c872b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b97b0316c48b49d59504a9c55fbfd0d3, user_id=ad388b6821e64ae0a47f01f6cb679cdb, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:22.512895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8d9f8-b4b1-4edb-9226-533a3c872b45 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:22.610033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1104/2201] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:22 2022] PUT /identity/v3/projects/b97b0316c48b49d59504a9c55fbfd0d3/users/ad388b6821e64ae0a47f01f6cb679cdb/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:22.616480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb686e0-fa0f-4cf3-87c4-10be6b905e6c admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:22.628873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:22.637061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb686e0-fa0f-4cf3-87c4-10be6b905e6c admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:22.772416 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64ba47-b257-4a15-a474-d530067b0d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:22.775684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ba47-b257-4a15-a474-d530067b0d11 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:22.776209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ba47-b257-4a15-a474-d530067b0d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:22.778221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ba47-b257-4a15-a474-d530067b0d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:22.779051 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64ba47-b257-4a15-a474-d530067b0d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:22.780818 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64ba47-b257-4a15-a474-d530067b0d11 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:22.802050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1098/2202] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:53:22.806375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8d9f8-b4b1-4edb-9226-533a3c872b45 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:22.818173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8d9f8-b4b1-4edb-9226-533a3c872b45 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:22.825358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd81614-30a1-4bb4-bf7d-5436f643bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:22.826673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd81614-30a1-4bb4-bf7d-5436f643bef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:22.827178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd81614-30a1-4bb4-bf7d-5436f643bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:22.828110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd81614-30a1-4bb4-bf7d-5436f643bef4 admin admin] PATH_INFO: `/v3/projects/b97b0316c48b49d59504a9c55fbfd0d3/users/ad388b6821e64ae0a47f01f6cb679cdb/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:22.831015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd81614-30a1-4bb4-bf7d-5436f643bef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b97b0316c48b49d59504a9c55fbfd0d3, user_id=ad388b6821e64ae0a47f01f6cb679cdb, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:22.835243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd81614-30a1-4bb4-bf7d-5436f643bef4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:22.850220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1105/2203] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:22 2022] PUT /identity/v3/projects/b97b0316c48b49d59504a9c55fbfd0d3/users/ad388b6821e64ae0a47f01f6cb679cdb/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:22.861599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb88d-51e5-40c0-a366-9da4310bc5c1 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:22.861986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb88d-51e5-40c0-a366-9da4310bc5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:22.862371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb88d-51e5-40c0-a366-9da4310bc5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:22.894521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-ad7eb88d-51e5-40c0-a366-9da4310bc5c1 None None] MFA Rules not processed for user `ad388b6821e64ae0a47f01f6cb679cdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:22.895702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ad7eb88d-51e5-40c0-a366-9da4310bc5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:23.003293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1099/2204] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:23.009182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd81614-30a1-4bb4-bf7d-5436f643bef4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:23.014345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-dcd81614-30a1-4bb4-bf7d-5436f643bef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:23.038292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:23.157231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd81614-30a1-4bb4-bf7d-5436f643bef4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:23.163696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbf203-6577-4603-ab3a-ae9233c28de2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:23.164537 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf203-6577-4603-ab3a-ae9233c28de2 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:23.164917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf203-6577-4603-ab3a-ae9233c28de2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:23.165419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf203-6577-4603-ab3a-ae9233c28de2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:23.181939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbf203-6577-4603-ab3a-ae9233c28de2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:23.187086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbf203-6577-4603-ab3a-ae9233c28de2 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:23.207225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1106/2205] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:53:23.305114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eb88d-51e5-40c0-a366-9da4310bc5c1 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:23.447547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eb88d-51e5-40c0-a366-9da4310bc5c1 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:23.454529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:23.455474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:23.455950 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:23.456383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] PATH_INFO: `/v3/users/7452d85fe48b4c37a5bb14cd186e7e71` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:23.458335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7452d85fe48b4c37a5bb14cd186e7e71)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:23.460802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:23.800375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7452d85fe48b4c37a5bb14cd186e7e71'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:23.821569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7452d85fe48b4c37a5bb14cd186e7e71'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:23.831818 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7452d85fe48b4c37a5bb14cd186e7e71'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:23.894275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7452d85fe48b4c37a5bb14cd186e7e71'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:23.896130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7452d85fe48b4c37a5bb14cd186e7e71'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:23.905431 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1100/2206] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:23 2022] DELETE /identity/v3/users/7452d85fe48b4c37a5bb14cd186e7e71 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:23.912150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd897fe8-f483-4aea-8251-fda0745120b0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:23.912683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd897fe8-f483-4aea-8251-fda0745120b0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:23.913137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd897fe8-f483-4aea-8251-fda0745120b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:23.963725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:24.026789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-dd897fe8-f483-4aea-8251-fda0745120b0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:24.031611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-dd897fe8-f483-4aea-8251-fda0745120b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:24.230853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1107/2207] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:24.255594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:24.257975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:24.467722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192cf96-c3d5-4507-9f7d-9ab0e4f5291e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:24.475271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed13a7e-1ba9-4296-add4-4db3a6d31f2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:24.476582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed13a7e-1ba9-4296-add4-4db3a6d31f2b service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:24.477078 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed13a7e-1ba9-4296-add4-4db3a6d31f2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:24.477568 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed13a7e-1ba9-4296-add4-4db3a6d31f2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:24.504858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed13a7e-1ba9-4296-add4-4db3a6d31f2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:24.507201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed13a7e-1ba9-4296-add4-4db3a6d31f2b service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:24.547339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1101/2208] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:24.662272 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14c136-333f-4000-bdf5-d7321fd8f8d7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:24.663194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14c136-333f-4000-bdf5-d7321fd8f8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:24.664015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14c136-333f-4000-bdf5-d7321fd8f8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:24.760117 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1e14c136-333f-4000-bdf5-d7321fd8f8d7 None None] MFA Rules not processed for user `27f78b50d76341e08e184582e8400a04`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:24.764230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1e14c136-333f-4000-bdf5-d7321fd8f8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:24.941989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1108/2209] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:24.947059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed13a7e-1ba9-4296-add4-4db3a6d31f2b service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:24.951949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7ed13a7e-1ba9-4296-add4-4db3a6d31f2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:24.981925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed13a7e-1ba9-4296-add4-4db3a6d31f2b service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:24.991171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7c673-fc4e-493f-8077-d27e5ca4e84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:24.993586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7c673-fc4e-493f-8077-d27e5ca4e84e admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:24.993586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7c673-fc4e-493f-8077-d27e5ca4e84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:24.993586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7c673-fc4e-493f-8077-d27e5ca4e84e admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:24.996234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7c673-fc4e-493f-8077-d27e5ca4e84e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:25.003911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7c673-fc4e-493f-8077-d27e5ca4e84e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:25.134081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1102/2210] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:24 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:25.135810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14c136-333f-4000-bdf5-d7321fd8f8d7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:25.188479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14c136-333f-4000-bdf5-d7321fd8f8d7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:25.208103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:25.429247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d51a50-9354-4005-94ec-b95bc45cdcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:25.430167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d51a50-9354-4005-94ec-b95bc45cdcf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:25.430542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d51a50-9354-4005-94ec-b95bc45cdcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:25.430828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d51a50-9354-4005-94ec-b95bc45cdcf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:25.431690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d51a50-9354-4005-94ec-b95bc45cdcf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:25.435588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d51a50-9354-4005-94ec-b95bc45cdcf3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:25.485682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7c673-fc4e-493f-8077-d27e5ca4e84e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:25.488828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8fc7c673-fc4e-493f-8077-d27e5ca4e84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:25.534763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7c673-fc4e-493f-8077-d27e5ca4e84e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:25.543589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:25.544312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:25.544672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:25.545030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] PATH_INFO: `/v3/projects/602ff95e8e44428cac7747abc1918c1d` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:25.546555 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=602ff95e8e44428cac7747abc1918c1d)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:25.549500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:25.596515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1109/2211] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:25 2022] POST /identity/v3/users => generated 299 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:25.604039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d51a50-9354-4005-94ec-b95bc45cdcf3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:25.634597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d51a50-9354-4005-94ec-b95bc45cdcf3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:25.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d51a3-7b8a-458e-a210-01b127c827bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:25.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d51a3-7b8a-458e-a210-01b127c827bc admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:25.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d51a3-7b8a-458e-a210-01b127c827bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:25.648168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d51a3-7b8a-458e-a210-01b127c827bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:25.653638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d51a3-7b8a-458e-a210-01b127c827bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:25.653638 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d51a3-7b8a-458e-a210-01b127c827bc admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:25.674649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1110/2212] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:25.680112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d51a3-7b8a-458e-a210-01b127c827bc admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:25.710629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d51a3-7b8a-458e-a210-01b127c827bc admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:25.735910 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e960af9-e176-4980-a779-644fb3290c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:25.737581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e960af9-e176-4980-a779-644fb3290c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:25.738549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e960af9-e176-4980-a779-644fb3290c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:25.739038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e960af9-e176-4980-a779-644fb3290c58 admin admin] PATH_INFO: `/v3/projects/c487dbdcde1f4ef098cd8bab7a1b7e27/users/31df9500e6d3467c9e944be3c368e77b/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:25.769283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e960af9-e176-4980-a779-644fb3290c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c487dbdcde1f4ef098cd8bab7a1b7e27, user_id=31df9500e6d3467c9e944be3c368e77b, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:25.772729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e960af9-e176-4980-a779-644fb3290c58 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:25.802277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '602ff95e8e44428cac7747abc1918c1d', 'request_id': 'req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:25.872930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1111/2213] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:25 2022] PUT /identity/v3/projects/c487dbdcde1f4ef098cd8bab7a1b7e27/users/31df9500e6d3467c9e944be3c368e77b/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:25.879465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e960af9-e176-4980-a779-644fb3290c58 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:25.887783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '602ff95e8e44428cac7747abc1918c1d', 'request_id': 'req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:25.895862 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '602ff95e8e44428cac7747abc1918c1d', 'request_id': 'req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:25.927501 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] The token cache is being invalidate because project 602ff95e8e44428cac7747abc1918c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:25.928020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:25.929441 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:25.932192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1103/2214] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:25 2022] DELETE /identity/v3/projects/602ff95e8e44428cac7747abc1918c1d => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:25.939763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:25.942744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:25.975168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:26.027860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e960af9-e176-4980-a779-644fb3290c58 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:26.038789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed603b60-5599-4ea8-b065-fd562f10dee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:26.039893 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed603b60-5599-4ea8-b065-fd562f10dee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:26.040598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed603b60-5599-4ea8-b065-fd562f10dee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:26.041104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed603b60-5599-4ea8-b065-fd562f10dee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:26.042336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed603b60-5599-4ea8-b065-fd562f10dee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:26.045895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed603b60-5599-4ea8-b065-fd562f10dee3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:26.067514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1112/2215] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:53:26.074396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed603b60-5599-4ea8-b065-fd562f10dee3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:26.077438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ed603b60-5599-4ea8-b065-fd562f10dee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:26.096101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed603b60-5599-4ea8-b065-fd562f10dee3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:26.101365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cac8d-fac5-4324-a18e-fffe8d3be7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:26.102312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cac8d-fac5-4324-a18e-fffe8d3be7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:26.102628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cac8d-fac5-4324-a18e-fffe8d3be7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:26.102909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cac8d-fac5-4324-a18e-fffe8d3be7e6 admin admin] PATH_INFO: `/v3/projects/c487dbdcde1f4ef098cd8bab7a1b7e27/users/31df9500e6d3467c9e944be3c368e77b/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:26.105004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cac8d-fac5-4324-a18e-fffe8d3be7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c487dbdcde1f4ef098cd8bab7a1b7e27, user_id=31df9500e6d3467c9e944be3c368e77b, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:26.108235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cac8d-fac5-4324-a18e-fffe8d3be7e6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:26.120127 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1113/2216] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:26 2022] PUT /identity/v3/projects/c487dbdcde1f4ef098cd8bab7a1b7e27/users/31df9500e6d3467c9e944be3c368e77b/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:26.145651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3493a-2c4b-40f5-be70-7fe90cb8dfa4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:26.254693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d5d85-325d-4d93-8ae9-e7d11f050f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:26.255832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d5d85-325d-4d93-8ae9-e7d11f050f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:26.256277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d5d85-325d-4d93-8ae9-e7d11f050f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:26.268565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d5d85-325d-4d93-8ae9-e7d11f050f2f admin admin] PATH_INFO: `/v3/users/ad388b6821e64ae0a47f01f6cb679cdb` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:26.270952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d5d85-325d-4d93-8ae9-e7d11f050f2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad388b6821e64ae0a47f01f6cb679cdb)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:26.274550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d5d85-325d-4d93-8ae9-e7d11f050f2f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:26.503234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-303d5d85-325d-4d93-8ae9-e7d11f050f2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad388b6821e64ae0a47f01f6cb679cdb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:26.522539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-303d5d85-325d-4d93-8ae9-e7d11f050f2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad388b6821e64ae0a47f01f6cb679cdb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:26.532360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-303d5d85-325d-4d93-8ae9-e7d11f050f2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad388b6821e64ae0a47f01f6cb679cdb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:26.638264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-303d5d85-325d-4d93-8ae9-e7d11f050f2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad388b6821e64ae0a47f01f6cb679cdb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:26.639458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-303d5d85-325d-4d93-8ae9-e7d11f050f2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad388b6821e64ae0a47f01f6cb679cdb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:26.666312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1104/2217] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:25 2022] DELETE /identity/v3/users/ad388b6821e64ae0a47f01f6cb679cdb => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:27.414065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cac8d-fac5-4324-a18e-fffe8d3be7e6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:27.423357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-415cac8d-fac5-4324-a18e-fffe8d3be7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:27.469146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cac8d-fac5-4324-a18e-fffe8d3be7e6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:27.543277 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:27.561206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:27.561206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:27.561206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:27.561206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] PATH_INFO: `/v3/projects/b97b0316c48b49d59504a9c55fbfd0d3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:27.561633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b97b0316c48b49d59504a9c55fbfd0d3)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:27.562519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:27.752186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b97b0316c48b49d59504a9c55fbfd0d3', 'request_id': 'req-9b41ff63-48e2-48da-b462-2f893b9b8eb1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:27.758533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b97b0316c48b49d59504a9c55fbfd0d3', 'request_id': 'req-9b41ff63-48e2-48da-b462-2f893b9b8eb1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:27.871120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b97b0316c48b49d59504a9c55fbfd0d3', 'request_id': 'req-9b41ff63-48e2-48da-b462-2f893b9b8eb1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:27.907476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] The token cache is being invalidate because project b97b0316c48b49d59504a9c55fbfd0d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:27.907950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:27.909586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9b41ff63-48e2-48da-b462-2f893b9b8eb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:27.911232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1114/2218] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:27 2022] DELETE /identity/v3/projects/b97b0316c48b49d59504a9c55fbfd0d3 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:27.916765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e9bc7-4130-4e52-8181-311c23c1ff1f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:27.918203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e9bc7-4130-4e52-8181-311c23c1ff1f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:27.918723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e9bc7-4130-4e52-8181-311c23c1ff1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:27.988261 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-6a8e9bc7-4130-4e52-8181-311c23c1ff1f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:27.989572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6a8e9bc7-4130-4e52-8181-311c23c1ff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:28.075186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:28.128732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1105/2219] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:28.137757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfcceb-6e23-4f77-b61b-07bc313f1e2a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:28.138787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfcceb-6e23-4f77-b61b-07bc313f1e2a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:28.139362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfcceb-6e23-4f77-b61b-07bc313f1e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:28.178269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-64dfcceb-6e23-4f77-b61b-07bc313f1e2a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:28.179838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-64dfcceb-6e23-4f77-b61b-07bc313f1e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:28.203322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1115/2220] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:28.205018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e9bc7-4130-4e52-8181-311c23c1ff1f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:28.223195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e9bc7-4130-4e52-8181-311c23c1ff1f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:28.244711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffade6-0dc1-4a74-9c55-633471e9ccce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:28.248438 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffade6-0dc1-4a74-9c55-633471e9ccce admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:28.248832 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffade6-0dc1-4a74-9c55-633471e9ccce admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:28.249314 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffade6-0dc1-4a74-9c55-633471e9ccce admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:28.255659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffade6-0dc1-4a74-9c55-633471e9ccce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:53:28.256347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffade6-0dc1-4a74-9c55-633471e9ccce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:28.266908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffade6-0dc1-4a74-9c55-633471e9ccce admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:28.291931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1106/2221] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:53:28.293266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfcceb-6e23-4f77-b61b-07bc313f1e2a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:28.306111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfcceb-6e23-4f77-b61b-07bc313f1e2a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:28.314910 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4b4b-44bc-456a-b50a-9596a780dee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:28.315614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4b4b-44bc-456a-b50a-9596a780dee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:28.316030 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4b4b-44bc-456a-b50a-9596a780dee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:28.317013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4b4b-44bc-456a-b50a-9596a780dee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:28.317826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d4b4b-44bc-456a-b50a-9596a780dee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:28.321420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d4b4b-44bc-456a-b50a-9596a780dee3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:28.453520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1116/2222] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:28 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:28.457412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffade6-0dc1-4a74-9c55-633471e9ccce admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:28.472489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffade6-0dc1-4a74-9c55-633471e9ccce admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:28.579364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d239e-b25b-4a7e-b7da-1a1cacff80c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:28.580074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d239e-b25b-4a7e-b7da-1a1cacff80c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:28.580333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d239e-b25b-4a7e-b7da-1a1cacff80c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:28.580606 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d239e-b25b-4a7e-b7da-1a1cacff80c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:28.581426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d239e-b25b-4a7e-b7da-1a1cacff80c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:28.584453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d239e-b25b-4a7e-b7da-1a1cacff80c3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:28.696824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1107/2223] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:28 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:53:28.703912 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4b4b-44bc-456a-b50a-9596a780dee3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:28.719740 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4b4b-44bc-456a-b50a-9596a780dee3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:28.731344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9f0ff-89b9-48d4-9e24-3a2fa9aeb21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:28.732675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9f0ff-89b9-48d4-9e24-3a2fa9aeb21f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:28.733191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9f0ff-89b9-48d4-9e24-3a2fa9aeb21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:28.733669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9f0ff-89b9-48d4-9e24-3a2fa9aeb21f admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:28.735082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9f0ff-89b9-48d4-9e24-3a2fa9aeb21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:28.738365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9f0ff-89b9-48d4-9e24-3a2fa9aeb21f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:28.764486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d239e-b25b-4a7e-b7da-1a1cacff80c3 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:28.773059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1117/2224] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:53:28.782517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d239e-b25b-4a7e-b7da-1a1cacff80c3 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:28.791134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3dc4d-7bbd-4484-8fd3-eaf37ef2af60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:28.791806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3dc4d-7bbd-4484-8fd3-eaf37ef2af60 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:28.792126 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3dc4d-7bbd-4484-8fd3-eaf37ef2af60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:28.793048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3dc4d-7bbd-4484-8fd3-eaf37ef2af60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:28.794125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3dc4d-7bbd-4484-8fd3-eaf37ef2af60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:28.797757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3dc4d-7bbd-4484-8fd3-eaf37ef2af60 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:28.832996 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9f0ff-89b9-48d4-9e24-3a2fa9aeb21f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:28.838646 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1108/2225] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:28.857957 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9f0ff-89b9-48d4-9e24-3a2fa9aeb21f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:28.876928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4694dbb-4596-4c40-b94c-673f5a1abfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:28.877407 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4694dbb-4596-4c40-b94c-673f5a1abfbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:28.877693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4694dbb-4596-4c40-b94c-673f5a1abfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:28.878124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4694dbb-4596-4c40-b94c-673f5a1abfbd admin admin] PATH_INFO: `/v3/projects/e69f6edc7acc4b1199b6a0de01242642/users/38981f544a074f9faa5b42d5247b70ba/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:28.924708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4694dbb-4596-4c40-b94c-673f5a1abfbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69f6edc7acc4b1199b6a0de01242642, user_id=38981f544a074f9faa5b42d5247b70ba, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:28.928370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4694dbb-4596-4c40-b94c-673f5a1abfbd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:29.031088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1118/2226] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:28 2022] PUT /identity/v3/projects/e69f6edc7acc4b1199b6a0de01242642/users/38981f544a074f9faa5b42d5247b70ba/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:29.039971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747164c-d580-4385-bee7-30b75a324b99 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:29.040723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747164c-d580-4385-bee7-30b75a324b99 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:29.040723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747164c-d580-4385-bee7-30b75a324b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:29.076274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-3747164c-d580-4385-bee7-30b75a324b99 None None] MFA Rules not processed for user `38981f544a074f9faa5b42d5247b70ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:29.078048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3747164c-d580-4385-bee7-30b75a324b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:29.267795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1109/2227] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:53:29.273445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4694dbb-4596-4c40-b94c-673f5a1abfbd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:29.457679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4694dbb-4596-4c40-b94c-673f5a1abfbd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:29.475593 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f148fdf-8c06-4dd1-9e2e-3843f14a9ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:29.476584 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f148fdf-8c06-4dd1-9e2e-3843f14a9ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:29.478368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f148fdf-8c06-4dd1-9e2e-3843f14a9ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:29.480384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f148fdf-8c06-4dd1-9e2e-3843f14a9ca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:29.481505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f148fdf-8c06-4dd1-9e2e-3843f14a9ca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:29.487265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f148fdf-8c06-4dd1-9e2e-3843f14a9ca0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:29.642043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:29.651713 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1119/2228] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:29 2022] POST /identity/v3/projects => generated 381 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:29.656055 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747164c-d580-4385-bee7-30b75a324b99 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:29.680896 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747164c-d580-4385-bee7-30b75a324b99 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:29.729199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:29.778977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3db6de-cde1-4020-b878-56c686fcddc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:29.781054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db6de-cde1-4020-b878-56c686fcddc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:29.782623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db6de-cde1-4020-b878-56c686fcddc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:29.784322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db6de-cde1-4020-b878-56c686fcddc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:29.785594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3db6de-cde1-4020-b878-56c686fcddc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:29.792021 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3db6de-cde1-4020-b878-56c686fcddc9 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:29.952216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1110/2229] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:29 2022] POST /identity/v3/users => generated 297 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:29.952391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f148fdf-8c06-4dd1-9e2e-3843f14a9ca0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:29.972114 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f148fdf-8c06-4dd1-9e2e-3843f14a9ca0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:29.988044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0347870-fa22-46aa-9e44-102206e85bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:29.989085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0347870-fa22-46aa-9e44-102206e85bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:29.989610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0347870-fa22-46aa-9e44-102206e85bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:29.990097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0347870-fa22-46aa-9e44-102206e85bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:29.990980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0347870-fa22-46aa-9e44-102206e85bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:29.993145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0347870-fa22-46aa-9e44-102206e85bcb admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:30.028425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1120/2230] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:30.032251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3db6de-cde1-4020-b878-56c686fcddc9 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:30.069083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3db6de-cde1-4020-b878-56c686fcddc9 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:30.077245 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145e6ff-af94-42dc-b63c-b5c5907f2648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:30.081208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145e6ff-af94-42dc-b63c-b5c5907f2648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:30.081579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145e6ff-af94-42dc-b63c-b5c5907f2648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:30.081868 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145e6ff-af94-42dc-b63c-b5c5907f2648 admin admin] PATH_INFO: `/v3/projects/bc674c1421914ccab17679e9a3b250f3/users/2bf68f51a155410483209d1f9baa5c39/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:30.121790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145e6ff-af94-42dc-b63c-b5c5907f2648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc674c1421914ccab17679e9a3b250f3, user_id=2bf68f51a155410483209d1f9baa5c39, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:30.125071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145e6ff-af94-42dc-b63c-b5c5907f2648 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:30.269943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1111/2231] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:30 2022] PUT /identity/v3/projects/bc674c1421914ccab17679e9a3b250f3/users/2bf68f51a155410483209d1f9baa5c39/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:30.273759 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0347870-fa22-46aa-9e44-102206e85bcb admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:30.291892 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0347870-fa22-46aa-9e44-102206e85bcb admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:30.498690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aedca-2dd5-4e16-9fd0-33cf2940e7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:30.499380 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aedca-2dd5-4e16-9fd0-33cf2940e7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:30.499613 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aedca-2dd5-4e16-9fd0-33cf2940e7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:30.500044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aedca-2dd5-4e16-9fd0-33cf2940e7b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:30.500716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aedca-2dd5-4e16-9fd0-33cf2940e7b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:30.502399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aedca-2dd5-4e16-9fd0-33cf2940e7b2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:30.521010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1121/2232] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:30.521199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145e6ff-af94-42dc-b63c-b5c5907f2648 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:30.536655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145e6ff-af94-42dc-b63c-b5c5907f2648 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:30.550110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753bec4-2ee4-4217-b07c-fa0d1382192a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:30.565387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bec4-2ee4-4217-b07c-fa0d1382192a admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:30.565897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bec4-2ee4-4217-b07c-fa0d1382192a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:30.567292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bec4-2ee4-4217-b07c-fa0d1382192a admin admin] PATH_INFO: `/v3/projects/bc674c1421914ccab17679e9a3b250f3/users/2bf68f51a155410483209d1f9baa5c39/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:30.590942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753bec4-2ee4-4217-b07c-fa0d1382192a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc674c1421914ccab17679e9a3b250f3, user_id=2bf68f51a155410483209d1f9baa5c39, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:30.602654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753bec4-2ee4-4217-b07c-fa0d1382192a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:30.639642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1112/2233] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:30 2022] PUT /identity/v3/projects/bc674c1421914ccab17679e9a3b250f3/users/2bf68f51a155410483209d1f9baa5c39/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:30.650759 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31811a-df50-4773-87bd-83add24499b2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:30.651636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31811a-df50-4773-87bd-83add24499b2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:30.652428 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31811a-df50-4773-87bd-83add24499b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:30.724305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-dc31811a-df50-4773-87bd-83add24499b2 None None] MFA Rules not processed for user `2bf68f51a155410483209d1f9baa5c39`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:30.726069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-dc31811a-df50-4773-87bd-83add24499b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:30.846617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1122/2234] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:30.872351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753bec4-2ee4-4217-b07c-fa0d1382192a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:30.874852 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9753bec4-2ee4-4217-b07c-fa0d1382192a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:31.132596 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753bec4-2ee4-4217-b07c-fa0d1382192a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:31.139490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd311b-d0f5-4db5-aa04-588394f808ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:31.140699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd311b-d0f5-4db5-aa04-588394f808ec service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:31.141160 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd311b-d0f5-4db5-aa04-588394f808ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:31.141584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd311b-d0f5-4db5-aa04-588394f808ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:31.160357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd311b-d0f5-4db5-aa04-588394f808ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:31.162844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd311b-d0f5-4db5-aa04-588394f808ec service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:31.176747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1113/2235] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:31.519136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31811a-df50-4773-87bd-83add24499b2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:31.520802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-dc31811a-df50-4773-87bd-83add24499b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:31.566701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31811a-df50-4773-87bd-83add24499b2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:31.594725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e99fc1-6c93-4d36-b956-e470ada2ed1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:31.595834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e99fc1-6c93-4d36-b956-e470ada2ed1b service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:31.596324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e99fc1-6c93-4d36-b956-e470ada2ed1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:31.596797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e99fc1-6c93-4d36-b956-e470ada2ed1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:31.751420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e99fc1-6c93-4d36-b956-e470ada2ed1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:31.752924 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e99fc1-6c93-4d36-b956-e470ada2ed1b service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:31.808496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1123/2236] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:32.688440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd311b-d0f5-4db5-aa04-588394f808ec service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:32.931061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd311b-d0f5-4db5-aa04-588394f808ec service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:32.957514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:32.962169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:32.963506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:32.964985 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] PATH_INFO: `/v3/users/38981f544a074f9faa5b42d5247b70ba` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:32.970917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38981f544a074f9faa5b42d5247b70ba)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:32.982128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:33.114324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:33.223140 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38981f544a074f9faa5b42d5247b70ba'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:33.281031 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38981f544a074f9faa5b42d5247b70ba'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:33.298786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38981f544a074f9faa5b42d5247b70ba'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:33.373213 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38981f544a074f9faa5b42d5247b70ba'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:33.376465 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38981f544a074f9faa5b42d5247b70ba'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:33.384202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1114/2237] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:32 2022] DELETE /identity/v3/users/38981f544a074f9faa5b42d5247b70ba => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:33.391478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488f7a2-c929-49d9-a105-abd2c92b6abe None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:33.392264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488f7a2-c929-49d9-a105-abd2c92b6abe None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:33.392264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488f7a2-c929-49d9-a105-abd2c92b6abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:33.413673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:33.421870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5488f7a2-c929-49d9-a105-abd2c92b6abe None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:33.423707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5488f7a2-c929-49d9-a105-abd2c92b6abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:33.505709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1124/2238] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:33.521302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:33.523337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:33.678387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1ae44-2e12-4e08-aba6-145c4bf44b43 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:33.689022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e64ce6-fe71-4ffb-80dd-8bbb78b057bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:33.690940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e64ce6-fe71-4ffb-80dd-8bbb78b057bd service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:33.691219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e64ce6-fe71-4ffb-80dd-8bbb78b057bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:33.691961 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e64ce6-fe71-4ffb-80dd-8bbb78b057bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:33.722414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e64ce6-fe71-4ffb-80dd-8bbb78b057bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:33.728544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e64ce6-fe71-4ffb-80dd-8bbb78b057bd service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:33.786302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1115/2239] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:53:34.917548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488f7a2-c929-49d9-a105-abd2c92b6abe None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:34.920917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5488f7a2-c929-49d9-a105-abd2c92b6abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:34.952225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488f7a2-c929-49d9-a105-abd2c92b6abe None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:34.968394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:34.969400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:34.969887 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:34.970782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] PATH_INFO: `/v3/projects/e69f6edc7acc4b1199b6a0de01242642` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:34.972420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e69f6edc7acc4b1199b6a0de01242642)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:34.974712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:35.236457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e69f6edc7acc4b1199b6a0de01242642', 'request_id': 'req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:35.251914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e69f6edc7acc4b1199b6a0de01242642', 'request_id': 'req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:35.386745 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e69f6edc7acc4b1199b6a0de01242642', 'request_id': 'req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:35.419220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] The token cache is being invalidate because project e69f6edc7acc4b1199b6a0de01242642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:35.422080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:35.424181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ed61bfe7-20a0-4121-ae6b-ebb07eb180f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:35.428792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1125/2240] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:34 2022] DELETE /identity/v3/projects/e69f6edc7acc4b1199b6a0de01242642 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:35.434976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e64ce6-fe71-4ffb-80dd-8bbb78b057bd service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:35.457062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e64ce6-fe71-4ffb-80dd-8bbb78b057bd service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:35.590065 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:35.590742 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:35.601607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:35.601607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] PATH_INFO: `/v3/users/2bf68f51a155410483209d1f9baa5c39` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:35.601607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf68f51a155410483209d1f9baa5c39)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:35.601607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:36.174548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf68f51a155410483209d1f9baa5c39'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:36.178846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:36.208090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf68f51a155410483209d1f9baa5c39'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:36.235241 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf68f51a155410483209d1f9baa5c39'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:36.401563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf68f51a155410483209d1f9baa5c39'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:36.404535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf68f51a155410483209d1f9baa5c39'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:36.429271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1116/2241] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:35 2022] DELETE /identity/v3/users/2bf68f51a155410483209d1f9baa5c39 => generated 0 bytes in 988 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:36.724957 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01379d9-54bd-4a60-ac5b-67861db0bb3a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:36.725528 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01379d9-54bd-4a60-ac5b-67861db0bb3a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:36.725528 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01379d9-54bd-4a60-ac5b-67861db0bb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:36.805045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d01379d9-54bd-4a60-ac5b-67861db0bb3a None None] MFA Rules not processed for user `31df9500e6d3467c9e944be3c368e77b`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:36.807140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d01379d9-54bd-4a60-ac5b-67861db0bb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:36.947814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1126/2242] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:53:36.969630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:36.973931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:37.105685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01379d9-54bd-4a60-ac5b-67861db0bb3a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:37.107148 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d01379d9-54bd-4a60-ac5b-67861db0bb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:37.138617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:37.162473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658e157-2e93-47c3-86dc-b0786035afec admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:37.172803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e7206-d8ea-45a5-bc30-bed5913fc24e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a59337d20e4678b8a16ac35bffbdb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:37.175651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e7206-d8ea-45a5-bc30-bed5913fc24e service glance] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:37.175651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e7206-d8ea-45a5-bc30-bed5913fc24e service glance] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:37.175651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e7206-d8ea-45a5-bc30-bed5913fc24e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:37.178275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4a2e7206-d8ea-45a5-bc30-bed5913fc24e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:37.237793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01379d9-54bd-4a60-ac5b-67861db0bb3a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:37.253384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab909e1a-1afa-46cf-94be-b2ff94705c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:37.255397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab909e1a-1afa-46cf-94be-b2ff94705c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:37.255814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab909e1a-1afa-46cf-94be-b2ff94705c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:37.258509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab909e1a-1afa-46cf-94be-b2ff94705c3b admin admin] PATH_INFO: `/v3/users/cdb3bd9006c7497a821e4e5226f8ac01` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:37.263232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab909e1a-1afa-46cf-94be-b2ff94705c3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb3bd9006c7497a821e4e5226f8ac01)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:37.266990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab909e1a-1afa-46cf-94be-b2ff94705c3b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:37.342707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e7206-d8ea-45a5-bc30-bed5913fc24e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:37.351110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e7206-d8ea-45a5-bc30-bed5913fc24e service glance] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:37.377300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1117/2243] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:37.460130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e7206-d8ea-45a5-bc30-bed5913fc24e service glance] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:37.461647 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4a2e7206-d8ea-45a5-bc30-bed5913fc24e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:37.477099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e7206-d8ea-45a5-bc30-bed5913fc24e service glance] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:37.482505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:37.483162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:37.483619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:37.484384 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] PATH_INFO: `/v3/projects/bc674c1421914ccab17679e9a3b250f3` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:37.485995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc674c1421914ccab17679e9a3b250f3)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:37.488215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:37.554530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab909e1a-1afa-46cf-94be-b2ff94705c3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb3bd9006c7497a821e4e5226f8ac01'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:37.583089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab909e1a-1afa-46cf-94be-b2ff94705c3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb3bd9006c7497a821e4e5226f8ac01'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:37.614889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab909e1a-1afa-46cf-94be-b2ff94705c3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb3bd9006c7497a821e4e5226f8ac01'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:37.717391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc674c1421914ccab17679e9a3b250f3', 'request_id': 'req-205a4040-1129-4006-8c2e-13deb1bf1093'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:37.784890 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab909e1a-1afa-46cf-94be-b2ff94705c3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb3bd9006c7497a821e4e5226f8ac01'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:37.786904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab909e1a-1afa-46cf-94be-b2ff94705c3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb3bd9006c7497a821e4e5226f8ac01'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:37.799397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1127/2244] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:37 2022] DELETE /identity/v3/users/cdb3bd9006c7497a821e4e5226f8ac01 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:37.807382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e85b1-e52c-4ba8-8445-7bad860d92c6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:37.808062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e85b1-e52c-4ba8-8445-7bad860d92c6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:37.808542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e85b1-e52c-4ba8-8445-7bad860d92c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:37.881217 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f92e85b1-e52c-4ba8-8445-7bad860d92c6 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:37.882726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f92e85b1-e52c-4ba8-8445-7bad860d92c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:37.938117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc674c1421914ccab17679e9a3b250f3', 'request_id': 'req-205a4040-1129-4006-8c2e-13deb1bf1093'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:37.955012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc674c1421914ccab17679e9a3b250f3', 'request_id': 'req-205a4040-1129-4006-8c2e-13deb1bf1093'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:38.002363 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] The token cache is being invalidate because project bc674c1421914ccab17679e9a3b250f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:38.004121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:38.007955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-205a4040-1129-4006-8c2e-13deb1bf1093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:38.013897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1118/2245] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:37 2022] DELETE /identity/v3/projects/bc674c1421914ccab17679e9a3b250f3 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:38.020262 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738875d-5937-4630-bd02-5458d4fbcb14 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:38.021633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738875d-5937-4630-bd02-5458d4fbcb14 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:38.023166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738875d-5937-4630-bd02-5458d4fbcb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:38.038083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1128/2246] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:38.041400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8992d7-365a-4969-99d6-95744fde97c0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:38.045089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8992d7-365a-4969-99d6-95744fde97c0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:38.045650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8992d7-365a-4969-99d6-95744fde97c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:38.079911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-e738875d-5937-4630-bd02-5458d4fbcb14 None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:38.082348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e738875d-5937-4630-bd02-5458d4fbcb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:38.099953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0d8992d7-365a-4969-99d6-95744fde97c0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:38.106679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0d8992d7-365a-4969-99d6-95744fde97c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:38.132357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1129/2247] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:38.150214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab0aa-b0b8-4dfb-bedc-31c9edcdc0e1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:38.151504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab0aa-b0b8-4dfb-bedc-31c9edcdc0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:38.153031 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab0aa-b0b8-4dfb-bedc-31c9edcdc0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:38.200820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c3dab0aa-b0b8-4dfb-bedc-31c9edcdc0e1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:38.202379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c3dab0aa-b0b8-4dfb-bedc-31c9edcdc0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:38.224598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1130/2248] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:38.225796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dab0aa-b0b8-4dfb-bedc-31c9edcdc0e1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:38.228274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1119/2249] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:53:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:38.230296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738875d-5937-4630-bd02-5458d4fbcb14 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:38.233014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c3dab0aa-b0b8-4dfb-bedc-31c9edcdc0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:38.256538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738875d-5937-4630-bd02-5458d4fbcb14 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:38.275810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a09c2-5b6f-4add-903c-0eee64829946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:38.277455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a09c2-5b6f-4add-903c-0eee64829946 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:38.278141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a09c2-5b6f-4add-903c-0eee64829946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:38.282459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a09c2-5b6f-4add-903c-0eee64829946 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:38.282459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a09c2-5b6f-4add-903c-0eee64829946 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:53:38.282459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a09c2-5b6f-4add-903c-0eee64829946 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:38.287838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a09c2-5b6f-4add-903c-0eee64829946 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:38.339694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1120/2250] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:53:38.346243 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a09c2-5b6f-4add-903c-0eee64829946 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:38.376525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a09c2-5b6f-4add-903c-0eee64829946 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:38.408547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e9cb3-0f5a-47ce-893c-a028a4a19979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:38.412041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e9cb3-0f5a-47ce-893c-a028a4a19979 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:38.416472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e9cb3-0f5a-47ce-893c-a028a4a19979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:38.417570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e9cb3-0f5a-47ce-893c-a028a4a19979 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:38.421724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e9cb3-0f5a-47ce-893c-a028a4a19979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:38.433063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e9cb3-0f5a-47ce-893c-a028a4a19979 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:38.447458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:38.465342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dab0aa-b0b8-4dfb-bedc-31c9edcdc0e1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:38.469980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc597e45-d1f1-4ae7-9d75-0bdb3d46eeb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:38.471301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc597e45-d1f1-4ae7-9d75-0bdb3d46eeb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:38.471729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc597e45-d1f1-4ae7-9d75-0bdb3d46eeb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:38.472147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc597e45-d1f1-4ae7-9d75-0bdb3d46eeb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:38.517894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc597e45-d1f1-4ae7-9d75-0bdb3d46eeb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:38.520036 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc597e45-d1f1-4ae7-9d75-0bdb3d46eeb2 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:38.534674 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1131/2251] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:38.536183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc597e45-d1f1-4ae7-9d75-0bdb3d46eeb2 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:38.538001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fc597e45-d1f1-4ae7-9d75-0bdb3d46eeb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:38.575380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1121/2252] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:38 2022] POST /identity/v3/projects => generated 375 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:38.579690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e9cb3-0f5a-47ce-893c-a028a4a19979 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:38.656095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc597e45-d1f1-4ae7-9d75-0bdb3d46eeb2 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:38.682511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e9cb3-0f5a-47ce-893c-a028a4a19979 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:38.694469 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e1ab4-0b14-44a0-9a24-22b20e8a2693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:38.695092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1ab4-0b14-44a0-9a24-22b20e8a2693 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:38.695481 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1ab4-0b14-44a0-9a24-22b20e8a2693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:38.698463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1ab4-0b14-44a0-9a24-22b20e8a2693 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:38.701438 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e1ab4-0b14-44a0-9a24-22b20e8a2693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:38.706187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e1ab4-0b14-44a0-9a24-22b20e8a2693 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:38.811259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53f6bd-83dd-4aa8-8216-385cfecbb2ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:38.817754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53f6bd-83dd-4aa8-8216-385cfecbb2ba service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:38.817754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53f6bd-83dd-4aa8-8216-385cfecbb2ba service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:38.817754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53f6bd-83dd-4aa8-8216-385cfecbb2ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:38.868202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1122/2253] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:38 2022] POST /identity/v3/users => generated 288 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:38.870576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e1ab4-0b14-44a0-9a24-22b20e8a2693 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:38.905888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e1ab4-0b14-44a0-9a24-22b20e8a2693 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:38.923632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02772f-1d26-490f-a3a1-83a739ec768c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:38.924263 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02772f-1d26-490f-a3a1-83a739ec768c admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:38.925595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02772f-1d26-490f-a3a1-83a739ec768c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:38.926093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02772f-1d26-490f-a3a1-83a739ec768c admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:38.933561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02772f-1d26-490f-a3a1-83a739ec768c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:38.939536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:38.941696 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02772f-1d26-490f-a3a1-83a739ec768c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:38.966973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1123/2254] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:38.972863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02772f-1d26-490f-a3a1-83a739ec768c admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:38.975108 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53f6bd-83dd-4aa8-8216-385cfecbb2ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:38.979715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53f6bd-83dd-4aa8-8216-385cfecbb2ba service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:38.992696 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02772f-1d26-490f-a3a1-83a739ec768c admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:39.004590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d5858-5af0-41ee-83da-e4c60592eb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:39.005949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d5858-5af0-41ee-83da-e4c60592eb58 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:39.006612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d5858-5af0-41ee-83da-e4c60592eb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:39.007431 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d5858-5af0-41ee-83da-e4c60592eb58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:39.008928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d5858-5af0-41ee-83da-e4c60592eb58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:39.013050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d5858-5af0-41ee-83da-e4c60592eb58 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:39.015148 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1132/2255] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:53:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:53:39.074015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1124/2256] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:53:39.079626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53f6bd-83dd-4aa8-8216-385cfecbb2ba service swift] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:39.102899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53f6bd-83dd-4aa8-8216-385cfecbb2ba service swift] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:39.118110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df9ca6-16ed-469a-9b87-1d223dd4c8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:39.119129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df9ca6-16ed-469a-9b87-1d223dd4c8d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:39.119405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df9ca6-16ed-469a-9b87-1d223dd4c8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:39.123089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df9ca6-16ed-469a-9b87-1d223dd4c8d2 admin admin] PATH_INFO: `/v3/projects/cc99348cb55649a7a0b12ae47353f405/users/c6ed01aa108747558d13a83ce8fad776/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:39.186159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df9ca6-16ed-469a-9b87-1d223dd4c8d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc99348cb55649a7a0b12ae47353f405, user_id=c6ed01aa108747558d13a83ce8fad776, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:39.190486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df9ca6-16ed-469a-9b87-1d223dd4c8d2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:39.376128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1133/2257] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:39 2022] PUT /identity/v3/projects/cc99348cb55649a7a0b12ae47353f405/users/c6ed01aa108747558d13a83ce8fad776/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:39.378699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90e5dc-960f-4a62-80a7-44b87f4cc308 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:39.379352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90e5dc-960f-4a62-80a7-44b87f4cc308 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:39.380050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90e5dc-960f-4a62-80a7-44b87f4cc308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:39.417612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-4b90e5dc-960f-4a62-80a7-44b87f4cc308 None None] MFA Rules not processed for user `c6ed01aa108747558d13a83ce8fad776`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:39.419376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4b90e5dc-960f-4a62-80a7-44b87f4cc308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:39.606452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1125/2258] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:39.612318 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df9ca6-16ed-469a-9b87-1d223dd4c8d2 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:39.758312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df9ca6-16ed-469a-9b87-1d223dd4c8d2 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:39.776911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80d3c5-afb5-478c-96be-8e7cd46c76c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:39.777250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80d3c5-afb5-478c-96be-8e7cd46c76c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:39.777419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80d3c5-afb5-478c-96be-8e7cd46c76c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:39.777567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80d3c5-afb5-478c-96be-8e7cd46c76c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:39.778265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80d3c5-afb5-478c-96be-8e7cd46c76c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:39.780477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80d3c5-afb5-478c-96be-8e7cd46c76c6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:39.920153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1134/2259] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:39 2022] POST /identity/v3/projects => generated 377 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:39.920794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90e5dc-960f-4a62-80a7-44b87f4cc308 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:39.936112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90e5dc-960f-4a62-80a7-44b87f4cc308 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:40.090990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f7fea-6892-4aa1-967c-e98eb6097abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:40.092730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f7fea-6892-4aa1-967c-e98eb6097abf admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:40.093340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f7fea-6892-4aa1-967c-e98eb6097abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:40.093865 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f7fea-6892-4aa1-967c-e98eb6097abf admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:40.094823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f7fea-6892-4aa1-967c-e98eb6097abf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:40.099936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f7fea-6892-4aa1-967c-e98eb6097abf admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:40.248804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1126/2260] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:39 2022] POST /identity/v3/users => generated 289 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:40.250323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80d3c5-afb5-478c-96be-8e7cd46c76c6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:40.269490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:40.279122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80d3c5-afb5-478c-96be-8e7cd46c76c6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:40.289051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b147535-d0cd-4ed1-b5df-d29b0b80c75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:40.291149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b147535-d0cd-4ed1-b5df-d29b0b80c75f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:40.292098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b147535-d0cd-4ed1-b5df-d29b0b80c75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:40.301338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b147535-d0cd-4ed1-b5df-d29b0b80c75f admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:40.302100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b147535-d0cd-4ed1-b5df-d29b0b80c75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:40.310946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b147535-d0cd-4ed1-b5df-d29b0b80c75f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:40.332082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1135/2261] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:40.337899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f7fea-6892-4aa1-967c-e98eb6097abf admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:40.357252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f7fea-6892-4aa1-967c-e98eb6097abf admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:40.454082 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52334135-9734-44b5-8d8d-caa5c3acb7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:40.456468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52334135-9734-44b5-8d8d-caa5c3acb7ad admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:40.457527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52334135-9734-44b5-8d8d-caa5c3acb7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:40.458463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52334135-9734-44b5-8d8d-caa5c3acb7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:40.460233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52334135-9734-44b5-8d8d-caa5c3acb7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:40.472735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52334135-9734-44b5-8d8d-caa5c3acb7ad admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:40.653510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b147535-d0cd-4ed1-b5df-d29b0b80c75f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:40.653510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1b147535-d0cd-4ed1-b5df-d29b0b80c75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:40.654311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1127/2262] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:40.655123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b147535-d0cd-4ed1-b5df-d29b0b80c75f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:40.655123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:40.655123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:40.655123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:40.656002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] PATH_INFO: `/v3/projects/f88335e0c0ea4d7eb0ab447fb25f5178` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:40.656002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f88335e0c0ea4d7eb0ab447fb25f5178)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:40.656002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:40.656470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52334135-9734-44b5-8d8d-caa5c3acb7ad admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:40.664052 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52334135-9734-44b5-8d8d-caa5c3acb7ad admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:40.677850 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672bf16-1fbf-4cbd-9512-8038710dcfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:40.678688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672bf16-1fbf-4cbd-9512-8038710dcfc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:40.682954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672bf16-1fbf-4cbd-9512-8038710dcfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:40.682954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672bf16-1fbf-4cbd-9512-8038710dcfc5 admin admin] PATH_INFO: `/v3/projects/66cd7a8305d247fd93521b59ecb3c91f/users/a259f7b04a8e4dc9a5ac5a5c42d4203a/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:40.705191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:40.723684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672bf16-1fbf-4cbd-9512-8038710dcfc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66cd7a8305d247fd93521b59ecb3c91f, user_id=a259f7b04a8e4dc9a5ac5a5c42d4203a, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:40.730668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672bf16-1fbf-4cbd-9512-8038710dcfc5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:41.091276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1128/2263] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:40 2022] PUT /identity/v3/projects/66cd7a8305d247fd93521b59ecb3c91f/users/a259f7b04a8e4dc9a5ac5a5c42d4203a/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:41.092851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c876a3b-4004-4ff0-991f-e9fed050595b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:41.093145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c876a3b-4004-4ff0-991f-e9fed050595b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:41.099765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c876a3b-4004-4ff0-991f-e9fed050595b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:41.123586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f88335e0c0ea4d7eb0ab447fb25f5178', 'request_id': 'req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:41.134126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f88335e0c0ea4d7eb0ab447fb25f5178', 'request_id': 'req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:41.149721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-1c876a3b-4004-4ff0-991f-e9fed050595b None None] MFA Rules not processed for user `34ef7ee27c3948ac9df1ff94ce24618e`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:41.151913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1c876a3b-4004-4ff0-991f-e9fed050595b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:41.241177 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f88335e0c0ea4d7eb0ab447fb25f5178', 'request_id': 'req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:41.284538 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] The token cache is being invalidate because project f88335e0c0ea4d7eb0ab447fb25f5178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:41.285202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:41.286074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3dd920e5-ebf7-4fee-9df8-4e29736d6bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:41.293370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1136/2264] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:40 2022] DELETE /identity/v3/projects/f88335e0c0ea4d7eb0ab447fb25f5178 => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:41.295513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d2186-1875-4943-8694-83180c89520e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:41.295974 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d2186-1875-4943-8694-83180c89520e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:41.296472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d2186-1875-4943-8694-83180c89520e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:41.353119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1129/2265] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:41.357982 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d92d2186-1875-4943-8694-83180c89520e None None] MFA Rules not processed for user `a259f7b04a8e4dc9a5ac5a5c42d4203a`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:41.359870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d92d2186-1875-4943-8694-83180c89520e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:41.363306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd7e40-0e67-4aac-9a03-56ee1c1f6f8c None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:41.363839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd7e40-0e67-4aac-9a03-56ee1c1f6f8c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:41.365456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd7e40-0e67-4aac-9a03-56ee1c1f6f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:41.420614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:41.444274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-9fbd7e40-0e67-4aac-9a03-56ee1c1f6f8c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:41.444274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9fbd7e40-0e67-4aac-9a03-56ee1c1f6f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:41.567044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1137/2266] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:41.568058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d2186-1875-4943-8694-83180c89520e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:41.575949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d92d2186-1875-4943-8694-83180c89520e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:41.621233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1130/2267] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:41.631024 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304318c-9b45-40cb-b914-fbf6dd2adc11 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:41.635787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304318c-9b45-40cb-b914-fbf6dd2adc11 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:41.635787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304318c-9b45-40cb-b914-fbf6dd2adc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:41.718517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-9304318c-9b45-40cb-b914-fbf6dd2adc11 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:41.721068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9304318c-9b45-40cb-b914-fbf6dd2adc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:41.748107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1131/2268] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:53:41.759170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304318c-9b45-40cb-b914-fbf6dd2adc11 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:41.767052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d2186-1875-4943-8694-83180c89520e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:41.771678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304318c-9b45-40cb-b914-fbf6dd2adc11 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:41.787205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df654d3-e679-41da-bcf7-c5f89d14fdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:41.787855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df654d3-e679-41da-bcf7-c5f89d14fdb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:41.787979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d32948-6994-4d7b-ae9c-858e7122f63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:41.788140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df654d3-e679-41da-bcf7-c5f89d14fdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:41.788416 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df654d3-e679-41da-bcf7-c5f89d14fdb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:41.789049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d32948-6994-4d7b-ae9c-858e7122f63c admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:41.789172 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df654d3-e679-41da-bcf7-c5f89d14fdb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:41.789386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d32948-6994-4d7b-ae9c-858e7122f63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:41.789744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d32948-6994-4d7b-ae9c-858e7122f63c admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:41.790629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d32948-6994-4d7b-ae9c-858e7122f63c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:53:41.791008 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d32948-6994-4d7b-ae9c-858e7122f63c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:41.791136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df654d3-e679-41da-bcf7-c5f89d14fdb1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:41.793732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d32948-6994-4d7b-ae9c-858e7122f63c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:41.857323 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1132/2269] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:53:41.862605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d32948-6994-4d7b-ae9c-858e7122f63c admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:41.940908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d32948-6994-4d7b-ae9c-858e7122f63c admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:41.953580 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1138/2270] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:41 2022] POST /identity/v3/projects => generated 377 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:41.957315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd7e4d-371f-4617-9e3b-a84f2845f713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:41.957315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd7e4d-371f-4617-9e3b-a84f2845f713 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:41.957315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd7e4d-371f-4617-9e3b-a84f2845f713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:41.957315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd7e4d-371f-4617-9e3b-a84f2845f713 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:41.957786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd7e4d-371f-4617-9e3b-a84f2845f713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:41.957871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df654d3-e679-41da-bcf7-c5f89d14fdb1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:41.959671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd7e4d-371f-4617-9e3b-a84f2845f713 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:42.029316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:42.139438 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1133/2271] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:53:41 2022] POST /identity/v3/projects => generated 397 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:42.141445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd7e4d-371f-4617-9e3b-a84f2845f713 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:42.145250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df654d3-e679-41da-bcf7-c5f89d14fdb1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:42.288055 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacf58e-ea7e-4bd4-bc22-15740bbbc064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:42.288603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacf58e-ea7e-4bd4-bc22-15740bbbc064 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:42.288884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacf58e-ea7e-4bd4-bc22-15740bbbc064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:42.289149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacf58e-ea7e-4bd4-bc22-15740bbbc064 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:42.297063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adacf58e-ea7e-4bd4-bc22-15740bbbc064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:42.300803 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adacf58e-ea7e-4bd4-bc22-15740bbbc064 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:42.432831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1139/2272] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:41 2022] POST /identity/v3/users => generated 295 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:42.439276 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacf58e-ea7e-4bd4-bc22-15740bbbc064 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:42.442945 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd7e4d-371f-4617-9e3b-a84f2845f713 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:42.463559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a3d64-302d-45d4-9f4d-40660d7045cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:42.466859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3d64-302d-45d4-9f4d-40660d7045cf admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:42.470876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3d64-302d-45d4-9f4d-40660d7045cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:42.473139 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacf58e-ea7e-4bd4-bc22-15740bbbc064 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:42.473858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3d64-302d-45d4-9f4d-40660d7045cf admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:42.476309 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a3d64-302d-45d4-9f4d-40660d7045cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:42.484551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a3d64-302d-45d4-9f4d-40660d7045cf admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:42.505204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ae818-ea8d-4940-b81f-fe4b0d9bd4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:42.508846 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ae818-ea8d-4940-b81f-fe4b0d9bd4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:42.511064 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:42.511463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ae818-ea8d-4940-b81f-fe4b0d9bd4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:42.511780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ae818-ea8d-4940-b81f-fe4b0d9bd4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:42.512589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ae818-ea8d-4940-b81f-fe4b0d9bd4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:42.514692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ae818-ea8d-4940-b81f-fe4b0d9bd4e7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:42.546612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1140/2273] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:42.546993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ae818-ea8d-4940-b81f-fe4b0d9bd4e7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:42.571993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ae818-ea8d-4940-b81f-fe4b0d9bd4e7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:42.588757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c6fd1-07c8-4a48-9cc5-c3542f13cd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:42.589692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c6fd1-07c8-4a48-9cc5-c3542f13cd7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:42.590129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c6fd1-07c8-4a48-9cc5-c3542f13cd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:42.590409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c6fd1-07c8-4a48-9cc5-c3542f13cd7c admin admin] PATH_INFO: `/v3/projects/0fb82de700ab49c68a2a95bb84bc2aec/users/f7b9f71704e74cd99352c7c74da6800d/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:42.632575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c6fd1-07c8-4a48-9cc5-c3542f13cd7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb82de700ab49c68a2a95bb84bc2aec, user_id=f7b9f71704e74cd99352c7c74da6800d, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:42.637188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c6fd1-07c8-4a48-9cc5-c3542f13cd7c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:42.683870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1134/2274] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:53:42 2022] POST /identity/v3/users => generated 299 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:53:42.688516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a3d64-302d-45d4-9f4d-40660d7045cf admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:42.723612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a3d64-302d-45d4-9f4d-40660d7045cf admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:42.740426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f8b5b-4750-4144-8812-0a4aa30a4c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:42.741120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8b5b-4750-4144-8812-0a4aa30a4c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:42.741678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8b5b-4750-4144-8812-0a4aa30a4c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:42.742118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8b5b-4750-4144-8812-0a4aa30a4c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:42.743055 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f8b5b-4750-4144-8812-0a4aa30a4c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:42.744159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1141/2275] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:42 2022] PUT /identity/v3/projects/0fb82de700ab49c68a2a95bb84bc2aec/users/f7b9f71704e74cd99352c7c74da6800d/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:42.749004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c6fd1-07c8-4a48-9cc5-c3542f13cd7c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:42.751292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f8b5b-4750-4144-8812-0a4aa30a4c4a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:42.780045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1135/2276] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:53:42.789034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f8b5b-4750-4144-8812-0a4aa30a4c4a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:42.805635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f8b5b-4750-4144-8812-0a4aa30a4c4a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:42.857363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c6fd1-07c8-4a48-9cc5-c3542f13cd7c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:42.863195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32b145-29c0-44d3-afe7-c1108b3b0d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:42.864187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32b145-29c0-44d3-afe7-c1108b3b0d9d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:42.867552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32b145-29c0-44d3-afe7-c1108b3b0d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:42.868161 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32b145-29c0-44d3-afe7-c1108b3b0d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:42.869079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32b145-29c0-44d3-afe7-c1108b3b0d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:42.871798 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32b145-29c0-44d3-afe7-c1108b3b0d9d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:42.892557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1142/2277] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:53:42.899206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32b145-29c0-44d3-afe7-c1108b3b0d9d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:42.915070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32b145-29c0-44d3-afe7-c1108b3b0d9d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:42.923328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b249aa-a7a3-445e-8481-4dfd3d49963a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:42.924452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b249aa-a7a3-445e-8481-4dfd3d49963a admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:42.924901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b249aa-a7a3-445e-8481-4dfd3d49963a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:42.925345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b249aa-a7a3-445e-8481-4dfd3d49963a admin admin] PATH_INFO: `/v3/projects/0fb82de700ab49c68a2a95bb84bc2aec/users/f7b9f71704e74cd99352c7c74da6800d/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:42.928427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b249aa-a7a3-445e-8481-4dfd3d49963a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb82de700ab49c68a2a95bb84bc2aec, user_id=f7b9f71704e74cd99352c7c74da6800d, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:42.931631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:42.934229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b249aa-a7a3-445e-8481-4dfd3d49963a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:42.962549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75f3b6-c24c-4b79-8c6a-1bf1c974d9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:42.962549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75f3b6-c24c-4b79-8c6a-1bf1c974d9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:42.962549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75f3b6-c24c-4b79-8c6a-1bf1c974d9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:42.962549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75f3b6-c24c-4b79-8c6a-1bf1c974d9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:42.963033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75f3b6-c24c-4b79-8c6a-1bf1c974d9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:42.964287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75f3b6-c24c-4b79-8c6a-1bf1c974d9e9 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:42.968542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1143/2278] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:42 2022] PUT /identity/v3/projects/0fb82de700ab49c68a2a95bb84bc2aec/users/f7b9f71704e74cd99352c7c74da6800d/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:42.970825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9203c3-04e7-45be-821e-00bcf4a6a4b4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:42.973704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9203c3-04e7-45be-821e-00bcf4a6a4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:42.973704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9203c3-04e7-45be-821e-00bcf4a6a4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:42.982318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1136/2279] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:53:42.986094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75f3b6-c24c-4b79-8c6a-1bf1c974d9e9 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:43.013956 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75f3b6-c24c-4b79-8c6a-1bf1c974d9e9 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:43.061008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8b9203c3-04e7-45be-821e-00bcf4a6a4b4 None None] MFA Rules not processed for user `f7b9f71704e74cd99352c7c74da6800d`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:43.064248 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8b9203c3-04e7-45be-821e-00bcf4a6a4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:43.221359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cec70f-ccf2-4acd-af5e-94851c03ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:43.222498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cec70f-ccf2-4acd-af5e-94851c03ffe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:43.227287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cec70f-ccf2-4acd-af5e-94851c03ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:43.227287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cec70f-ccf2-4acd-af5e-94851c03ffe2 admin admin] PATH_INFO: `/v3/projects/3f287407eb4c46a7aeb70df0c131d1f8/users/5718d26e902b47f090c85c270366f633/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:43.280444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cec70f-ccf2-4acd-af5e-94851c03ffe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f287407eb4c46a7aeb70df0c131d1f8, user_id=5718d26e902b47f090c85c270366f633, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:43.286211 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cec70f-ccf2-4acd-af5e-94851c03ffe2 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:43.304250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1144/2280] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:43.306295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1b231-f8f1-4044-be31-64cffdcf9e8a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:43.306637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1b231-f8f1-4044-be31-64cffdcf9e8a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:43.307151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1b231-f8f1-4044-be31-64cffdcf9e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:43.334397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f0c1b231-f8f1-4044-be31-64cffdcf9e8a None None] MFA Rules not processed for user `74d57012e9a04960b6c5dde2243dafbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:43.336482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f0c1b231-f8f1-4044-be31-64cffdcf9e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:43.383670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:43.420302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1137/2281] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:53:42 2022] PUT /identity/v3/projects/3f287407eb4c46a7aeb70df0c131d1f8/users/5718d26e902b47f090c85c270366f633/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:43.421466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cec70f-ccf2-4acd-af5e-94851c03ffe2 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:43.436876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-59cec70f-ccf2-4acd-af5e-94851c03ffe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:43.492470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1145/2282] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:43.509093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8529f7b-d614-4c2a-9418-0b97553872a0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:43.509400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8529f7b-d614-4c2a-9418-0b97553872a0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:43.509640 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8529f7b-d614-4c2a-9418-0b97553872a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:43.606486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cec70f-ccf2-4acd-af5e-94851c03ffe2 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:43.606486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ed864-0e32-44af-bc2b-4522622267c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:43.606486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ed864-0e32-44af-bc2b-4522622267c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:43.606486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ed864-0e32-44af-bc2b-4522622267c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:43.606984 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ed864-0e32-44af-bc2b-4522622267c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:43.606984 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ae5ed864-0e32-44af-bc2b-4522622267c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:43.624774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b8529f7b-d614-4c2a-9418-0b97553872a0 None None] MFA Rules not processed for user `5718d26e902b47f090c85c270366f633`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:43.632790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b8529f7b-d614-4c2a-9418-0b97553872a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:43.725773 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ed864-0e32-44af-bc2b-4522622267c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:43.727554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ed864-0e32-44af-bc2b-4522622267c9 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:43.748630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1138/2283] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:43.913634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1146/2284] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:53:43.930356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ed864-0e32-44af-bc2b-4522622267c9 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:43.932433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ae5ed864-0e32-44af-bc2b-4522622267c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:43.950856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ed864-0e32-44af-bc2b-4522622267c9 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:43.957843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ec012-1da1-4596-a838-c2a3838e57b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:43.958540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ec012-1da1-4596-a838-c2a3838e57b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:43.958799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ec012-1da1-4596-a838-c2a3838e57b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:43.959075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ec012-1da1-4596-a838-c2a3838e57b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:43.987996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ec012-1da1-4596-a838-c2a3838e57b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:43.989537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ec012-1da1-4596-a838-c2a3838e57b4 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:44.004069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1139/2285] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:43 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:46.901954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8529f7b-d614-4c2a-9418-0b97553872a0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:46.916426 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8529f7b-d614-4c2a-9418-0b97553872a0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:46.929409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefa4da-f651-4be5-9185-0f49d9df3d1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:46.930348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefa4da-f651-4be5-9185-0f49d9df3d1d service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:46.930750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefa4da-f651-4be5-9185-0f49d9df3d1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:46.931126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefa4da-f651-4be5-9185-0f49d9df3d1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:47.132089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefa4da-f651-4be5-9185-0f49d9df3d1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:47.144468 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefa4da-f651-4be5-9185-0f49d9df3d1d service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:47.201821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1147/2286] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:50.174123 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ec012-1da1-4596-a838-c2a3838e57b4 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:50.191234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ec012-1da1-4596-a838-c2a3838e57b4 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:50.199893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b6dea-e33f-417a-9aa6-548dda27d4ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:50.201071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b6dea-e33f-417a-9aa6-548dda27d4ad service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:50.201649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b6dea-e33f-417a-9aa6-548dda27d4ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:50.202343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b6dea-e33f-417a-9aa6-548dda27d4ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:50.234505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:50.399032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b6dea-e33f-417a-9aa6-548dda27d4ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:50.399032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b6dea-e33f-417a-9aa6-548dda27d4ad service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:50.426376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1140/2287] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:52.067821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb34b3a-a680-4d9f-8241-ef4dd1b4089a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:52.073929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb34b3a-a680-4d9f-8241-ef4dd1b4089a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:52.074214 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb34b3a-a680-4d9f-8241-ef4dd1b4089a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:52.130192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-cdb34b3a-a680-4d9f-8241-ef4dd1b4089a None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:52.131555 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cdb34b3a-a680-4d9f-8241-ef4dd1b4089a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:52.290340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1148/2288] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:53:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:52.310038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b6dea-e33f-417a-9aa6-548dda27d4ad service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:52.312914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-199b6dea-e33f-417a-9aa6-548dda27d4ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:52.542747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b6dea-e33f-417a-9aa6-548dda27d4ad service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:52.567368 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50d78e-2365-4776-a6ac-bfaefebc64a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:52.568277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50d78e-2365-4776-a6ac-bfaefebc64a4 service swift] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:52.568724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50d78e-2365-4776-a6ac-bfaefebc64a4 service swift] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:52.569182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50d78e-2365-4776-a6ac-bfaefebc64a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:52.599305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50d78e-2365-4776-a6ac-bfaefebc64a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:52.601004 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50d78e-2365-4776-a6ac-bfaefebc64a4 service swift] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:52.622086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1141/2289] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:53:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:53:55.117637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb34b3a-a680-4d9f-8241-ef4dd1b4089a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:55.129368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:55.522995 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb34b3a-a680-4d9f-8241-ef4dd1b4089a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:55.547387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:55.550823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:55.551706 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:55.554015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] PATH_INFO: `/v3/users/5718d26e902b47f090c85c270366f633` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:55.557564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5718d26e902b47f090c85c270366f633)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:55.567373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:55.875983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5718d26e902b47f090c85c270366f633'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:55.938257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5718d26e902b47f090c85c270366f633'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:55.961457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5718d26e902b47f090c85c270366f633'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:56.098126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5718d26e902b47f090c85c270366f633'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:56.099596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5718d26e902b47f090c85c270366f633'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:56.112557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1149/2290] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:55 2022] DELETE /identity/v3/users/5718d26e902b47f090c85c270366f633 => generated 0 bytes in 996 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:53:56.118212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554f992-a0c8-4987-a97f-2e7e6b3cfbb0 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:56.119494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554f992-a0c8-4987-a97f-2e7e6b3cfbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:56.120296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554f992-a0c8-4987-a97f-2e7e6b3cfbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:56.179853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-5554f992-a0c8-4987-a97f-2e7e6b3cfbb0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:56.191989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5554f992-a0c8-4987-a97f-2e7e6b3cfbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:56.454590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1142/2291] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:56.466627 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:56.475363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:56.577370 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577cd40b-f43a-4c97-b6d0-877f6e42e678 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:56.581583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf762d0c-0e54-4db6-8090-4ed18faaf13a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:56.582543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf762d0c-0e54-4db6-8090-4ed18faaf13a service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:56.583026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf762d0c-0e54-4db6-8090-4ed18faaf13a service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:56.583494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf762d0c-0e54-4db6-8090-4ed18faaf13a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:56.608113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf762d0c-0e54-4db6-8090-4ed18faaf13a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:56.614676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf762d0c-0e54-4db6-8090-4ed18faaf13a service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:56.639178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1150/2292] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:57.061789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554f992-a0c8-4987-a97f-2e7e6b3cfbb0 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:57.063784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5554f992-a0c8-4987-a97f-2e7e6b3cfbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:57.087711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554f992-a0c8-4987-a97f-2e7e6b3cfbb0 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:57.117333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:57.120466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:57.130486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:57.135565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] PATH_INFO: `/v3/users/c6ed01aa108747558d13a83ce8fad776` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:57.144517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6ed01aa108747558d13a83ce8fad776)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:57.156536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:57.341835 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:57.543687 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6ed01aa108747558d13a83ce8fad776'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:57.657964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6ed01aa108747558d13a83ce8fad776'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:57.657964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6ed01aa108747558d13a83ce8fad776'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:58.440402 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6ed01aa108747558d13a83ce8fad776'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:58.454488 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6ed01aa108747558d13a83ce8fad776'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:58.466155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1143/2293] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:53:57 2022] DELETE /identity/v3/users/c6ed01aa108747558d13a83ce8fad776 => generated 0 bytes in 1401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:53:58.467814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a575e6-2eef-44f5-af46-c3ddb817116d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:58.468369 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a575e6-2eef-44f5-af46-c3ddb817116d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:58.468369 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a575e6-2eef-44f5-af46-c3ddb817116d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:58.490056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-69a575e6-2eef-44f5-af46-c3ddb817116d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:53:58.491229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-69a575e6-2eef-44f5-af46-c3ddb817116d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:58.537233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:53:58.568115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1151/2294] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:53:58.587492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:58.589399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:58.695588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867ebd0-3ef3-4405-b4e4-8a3ae5ca9b28 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:58.700881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778c670-05d6-4710-a2fa-9530e1a34db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:58.701964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778c670-05d6-4710-a2fa-9530e1a34db9 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:58.702470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778c670-05d6-4710-a2fa-9530e1a34db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:58.705645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778c670-05d6-4710-a2fa-9530e1a34db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:58.721591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778c670-05d6-4710-a2fa-9530e1a34db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:58.723897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778c670-05d6-4710-a2fa-9530e1a34db9 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:58.735361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1144/2295] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:53:59.166560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a575e6-2eef-44f5-af46-c3ddb817116d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:53:59.168093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-69a575e6-2eef-44f5-af46-c3ddb817116d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:53:59.198910 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a575e6-2eef-44f5-af46-c3ddb817116d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:53:59.207848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:53:59.208532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:53:59.208818 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:53:59.209124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] PATH_INFO: `/v3/projects/3f287407eb4c46a7aeb70df0c131d1f8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:53:59.211950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f287407eb4c46a7aeb70df0c131d1f8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:53:59.214996 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:53:59.400373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f287407eb4c46a7aeb70df0c131d1f8', 'request_id': 'req-3a5000b7-af13-41c7-9317-33af5abf6e60'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:59.424062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f287407eb4c46a7aeb70df0c131d1f8', 'request_id': 'req-3a5000b7-af13-41c7-9317-33af5abf6e60'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:59.531044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f287407eb4c46a7aeb70df0c131d1f8', 'request_id': 'req-3a5000b7-af13-41c7-9317-33af5abf6e60'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:59.677287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] The token cache is being invalidate because project 3f287407eb4c46a7aeb70df0c131d1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:53:59.678067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:59.683393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:53:59.718752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1152/2296] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:53:59 2022] DELETE /identity/v3/projects/3f287407eb4c46a7aeb70df0c131d1f8 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:54:00.162219 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778c670-05d6-4710-a2fa-9530e1a34db9 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:00.164234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f778c670-05d6-4710-a2fa-9530e1a34db9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:00.201426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778c670-05d6-4710-a2fa-9530e1a34db9 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:00.396330 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:00.398621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:00.400786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:00.401742 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] PATH_INFO: `/v3/projects/cc99348cb55649a7a0b12ae47353f405` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:00.405695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc99348cb55649a7a0b12ae47353f405)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:00.410744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:00.569534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc99348cb55649a7a0b12ae47353f405', 'request_id': 'req-46fabfc9-e978-4631-ab40-02430b533f2e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:00.572821 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:54:00.824416 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc99348cb55649a7a0b12ae47353f405', 'request_id': 'req-46fabfc9-e978-4631-ab40-02430b533f2e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:00.833473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc99348cb55649a7a0b12ae47353f405', 'request_id': 'req-46fabfc9-e978-4631-ab40-02430b533f2e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:00.852361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] The token cache is being invalidate because project cc99348cb55649a7a0b12ae47353f405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:54:00.853322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:00.854229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-46fabfc9-e978-4631-ab40-02430b533f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:00.856549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1145/2297] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:54:00 2022] DELETE /identity/v3/projects/cc99348cb55649a7a0b12ae47353f405 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:54:00.861012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:00.863297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:00.885961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5000b7-af13-41c7-9317-33af5abf6e60 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:00.973450 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:00.974118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:00.974525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:00.974792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] PATH_INFO: `/v3/users/a259f7b04a8e4dc9a5ac5a5c42d4203a` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:00.976648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a259f7b04a8e4dc9a5ac5a5c42d4203a)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:00.979379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:01.138500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306ecde-98a5-464d-9c2c-4be86f4972f5 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:01.138777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306ecde-98a5-464d-9c2c-4be86f4972f5 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:01.139059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306ecde-98a5-464d-9c2c-4be86f4972f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:01.170350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-8306ecde-98a5-464d-9c2c-4be86f4972f5 None None] MFA Rules not processed for user `27f78b50d76341e08e184582e8400a04`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:01.172086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8306ecde-98a5-464d-9c2c-4be86f4972f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:01.248835 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1146/2298] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:54:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:54:01.329223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a259f7b04a8e4dc9a5ac5a5c42d4203a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:01.336675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:54:01.360167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a259f7b04a8e4dc9a5ac5a5c42d4203a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:01.373375 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a259f7b04a8e4dc9a5ac5a5c42d4203a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:01.463753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a259f7b04a8e4dc9a5ac5a5c42d4203a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:01.467462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a259f7b04a8e4dc9a5ac5a5c42d4203a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:01.485334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1153/2299] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:54:00 2022] DELETE /identity/v3/users/a259f7b04a8e4dc9a5ac5a5c42d4203a => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:54:02.285860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306ecde-98a5-464d-9c2c-4be86f4972f5 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:02.288155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8306ecde-98a5-464d-9c2c-4be86f4972f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:02.320485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306ecde-98a5-464d-9c2c-4be86f4972f5 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:02.426664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:02.427380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:02.427669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:02.427946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] PATH_INFO: `/v3/projects/66cd7a8305d247fd93521b59ecb3c91f` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:02.429294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66cd7a8305d247fd93521b59ecb3c91f)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:02.434084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:02.652072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66cd7a8305d247fd93521b59ecb3c91f', 'request_id': 'req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:02.763942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66cd7a8305d247fd93521b59ecb3c91f', 'request_id': 'req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:02.770667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66cd7a8305d247fd93521b59ecb3c91f', 'request_id': 'req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:02.800793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] The token cache is being invalidate because project 66cd7a8305d247fd93521b59ecb3c91f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:54:02.801149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:02.802153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:02.804377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1147/2300] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:54:02 2022] DELETE /identity/v3/projects/66cd7a8305d247fd93521b59ecb3c91f => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:54:02.810575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:02.813709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:02.837426 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa1bcd-6fd2-4025-bc16-3866579b08b4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:02.908864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ece7c8-92a5-4021-ace3-66335617a3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:02.909521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ece7c8-92a5-4021-ace3-66335617a3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:02.909798 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ece7c8-92a5-4021-ace3-66335617a3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:02.910066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ece7c8-92a5-4021-ace3-66335617a3b4 admin admin] PATH_INFO: `/v3/users/f7b9f71704e74cd99352c7c74da6800d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:02.911769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ece7c8-92a5-4021-ace3-66335617a3b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7b9f71704e74cd99352c7c74da6800d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:02.916540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ece7c8-92a5-4021-ace3-66335617a3b4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:03.258343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-12ece7c8-92a5-4021-ace3-66335617a3b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7b9f71704e74cd99352c7c74da6800d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:03.298349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-12ece7c8-92a5-4021-ace3-66335617a3b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7b9f71704e74cd99352c7c74da6800d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:03.312917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-12ece7c8-92a5-4021-ace3-66335617a3b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7b9f71704e74cd99352c7c74da6800d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:03.572886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-12ece7c8-92a5-4021-ace3-66335617a3b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7b9f71704e74cd99352c7c74da6800d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:03.572886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-12ece7c8-92a5-4021-ace3-66335617a3b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7b9f71704e74cd99352c7c74da6800d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:03.572886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1154/2301] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:54:02 2022] DELETE /identity/v3/users/f7b9f71704e74cd99352c7c74da6800d => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:54:04.816819 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:04.821833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:04.849420 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21437dbc-8bc0-4ea9-a5ac-b1681111e6f3 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:04.868325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:54:04.959940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:04.960971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:04.961340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:04.961746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] PATH_INFO: `/v3/projects/0fb82de700ab49c68a2a95bb84bc2aec` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:04.963408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb82de700ab49c68a2a95bb84bc2aec)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:04.967805 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:05.175052 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb82de700ab49c68a2a95bb84bc2aec', 'request_id': 'req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:05.296042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb82de700ab49c68a2a95bb84bc2aec', 'request_id': 'req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:05.304092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb82de700ab49c68a2a95bb84bc2aec', 'request_id': 'req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:05.324559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] The token cache is being invalidate because project 0fb82de700ab49c68a2a95bb84bc2aec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:54:05.335386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:05.335386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9d27b8c2-7782-4df3-b4ae-21cd6cc84ad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:05.335386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1148/2302] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:54:04 2022] DELETE /identity/v3/projects/0fb82de700ab49c68a2a95bb84bc2aec => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:54:05.335687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df0d16-063d-45e4-833a-7d4195c53a81 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:05.335687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df0d16-063d-45e4-833a-7d4195c53a81 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:05.336170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df0d16-063d-45e4-833a-7d4195c53a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:05.417358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-92df0d16-063d-45e4-833a-7d4195c53a81 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:05.419685 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-92df0d16-063d-45e4-833a-7d4195c53a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:05.574592 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:54:05.611060 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1155/2303] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:05.633028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2419-7523-4e9c-a043-fccabfefa2d9 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:05.636511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2419-7523-4e9c-a043-fccabfefa2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:05.636511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2419-7523-4e9c-a043-fccabfefa2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:05.691784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-74cb2419-7523-4e9c-a043-fccabfefa2d9 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:05.693299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-74cb2419-7523-4e9c-a043-fccabfefa2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:05.713313 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1149/2304] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:05.713571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df0d16-063d-45e4-833a-7d4195c53a81 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:05.732866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df0d16-063d-45e4-833a-7d4195c53a81 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:05.757807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f95fc-7b42-4f6e-a9b8-b2236b54da28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:05.760274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f95fc-7b42-4f6e-a9b8-b2236b54da28 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:05.766056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f95fc-7b42-4f6e-a9b8-b2236b54da28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:05.766056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f95fc-7b42-4f6e-a9b8-b2236b54da28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:05.766056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f95fc-7b42-4f6e-a9b8-b2236b54da28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:54:05.766056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f95fc-7b42-4f6e-a9b8-b2236b54da28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:05.769053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f95fc-7b42-4f6e-a9b8-b2236b54da28 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:05.804731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb2419-7523-4e9c-a043-fccabfefa2d9 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:05.832104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1156/2305] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:54:05.837349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb2419-7523-4e9c-a043-fccabfefa2d9 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:05.859247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba135a5-9657-4929-8c63-9bda2562ebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:05.860964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba135a5-9657-4929-8c63-9bda2562ebfc admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:05.861506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba135a5-9657-4929-8c63-9bda2562ebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:05.862030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba135a5-9657-4929-8c63-9bda2562ebfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:05.863547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba135a5-9657-4929-8c63-9bda2562ebfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:05.868846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba135a5-9657-4929-8c63-9bda2562ebfc admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:06.012009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1150/2306] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:54:05 2022] POST /identity/v3/projects => generated 367 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:54:06.012802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f95fc-7b42-4f6e-a9b8-b2236b54da28 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:06.029126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f95fc-7b42-4f6e-a9b8-b2236b54da28 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:06.192616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea1c31-9b94-4d24-847e-49d26aa1bd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:06.193425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea1c31-9b94-4d24-847e-49d26aa1bd2c admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:06.193827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea1c31-9b94-4d24-847e-49d26aa1bd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:06.194210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea1c31-9b94-4d24-847e-49d26aa1bd2c admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:06.195339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea1c31-9b94-4d24-847e-49d26aa1bd2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:06.197520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea1c31-9b94-4d24-847e-49d26aa1bd2c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:06.334102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1157/2307] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:54:06 2022] POST /identity/v3/users => generated 284 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:54:06.347028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba135a5-9657-4929-8c63-9bda2562ebfc admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:06.374618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba135a5-9657-4929-8c63-9bda2562ebfc admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:06.401699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef84984-e884-4cfa-97a3-3836633c1d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:06.402398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef84984-e884-4cfa-97a3-3836633c1d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:06.402718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef84984-e884-4cfa-97a3-3836633c1d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:06.403057 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef84984-e884-4cfa-97a3-3836633c1d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:06.403781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef84984-e884-4cfa-97a3-3836633c1d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:06.414302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef84984-e884-4cfa-97a3-3836633c1d07 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:06.440529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1151/2308] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:54:06.442081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea1c31-9b94-4d24-847e-49d26aa1bd2c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:06.485328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea1c31-9b94-4d24-847e-49d26aa1bd2c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:06.491857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02531f07-1f07-47df-b4c8-97ca0a9c94b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:06.492908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02531f07-1f07-47df-b4c8-97ca0a9c94b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:06.494059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02531f07-1f07-47df-b4c8-97ca0a9c94b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:06.494722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02531f07-1f07-47df-b4c8-97ca0a9c94b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:06.495622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02531f07-1f07-47df-b4c8-97ca0a9c94b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:06.497474 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02531f07-1f07-47df-b4c8-97ca0a9c94b0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:06.511834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1158/2309] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:54:06.526725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef84984-e884-4cfa-97a3-3836633c1d07 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:06.531433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef84984-e884-4cfa-97a3-3836633c1d07 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:06.538764 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4710e-0dee-4377-820a-c0936583baf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:06.539456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4710e-0dee-4377-820a-c0936583baf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:06.540272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4710e-0dee-4377-820a-c0936583baf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:06.540556 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4710e-0dee-4377-820a-c0936583baf9 admin admin] PATH_INFO: `/v3/projects/9e8c274268854032aade285167d8080e/users/cbb1d72d77ad4688bcbb567075ff556a/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:06.565001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4710e-0dee-4377-820a-c0936583baf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e8c274268854032aade285167d8080e, user_id=cbb1d72d77ad4688bcbb567075ff556a, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:06.568156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4710e-0dee-4377-820a-c0936583baf9 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:06.678747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1152/2310] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:54:06 2022] PUT /identity/v3/projects/9e8c274268854032aade285167d8080e/users/cbb1d72d77ad4688bcbb567075ff556a/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:54:06.684486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4c0f3-2852-4ee6-bfd5-7369b6e72145 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:06.684862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4c0f3-2852-4ee6-bfd5-7369b6e72145 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:06.685204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4c0f3-2852-4ee6-bfd5-7369b6e72145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:06.721042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-69b4c0f3-2852-4ee6-bfd5-7369b6e72145 None None] MFA Rules not processed for user `cbb1d72d77ad4688bcbb567075ff556a`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:06.722473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-69b4c0f3-2852-4ee6-bfd5-7369b6e72145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:06.885501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1159/2311] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:54:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:06.895724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4710e-0dee-4377-820a-c0936583baf9 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:06.898709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1fc4710e-0dee-4377-820a-c0936583baf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:06.965353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:54:06.991672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4710e-0dee-4377-820a-c0936583baf9 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:06.998064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7bff0-f268-4e94-a5ae-a6571e4d0c9e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a59337d20e4678b8a16ac35bffbdb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:06.999492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7bff0-f268-4e94-a5ae-a6571e4d0c9e service glance] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:07.000146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7bff0-f268-4e94-a5ae-a6571e4d0c9e service glance] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:07.000733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7bff0-f268-4e94-a5ae-a6571e4d0c9e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:07.013007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7bff0-f268-4e94-a5ae-a6571e4d0c9e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:07.015513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7bff0-f268-4e94-a5ae-a6571e4d0c9e service glance] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:07.033042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1153/2312] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:54:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:54:09.138770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4c0f3-2852-4ee6-bfd5-7369b6e72145 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:09.141261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-69b4c0f3-2852-4ee6-bfd5-7369b6e72145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:09.288859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:54:09.373842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4c0f3-2852-4ee6-bfd5-7369b6e72145 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:09.394587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecc578-9a2c-4a0a-99fb-7c78f5920b72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:09.395225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecc578-9a2c-4a0a-99fb-7c78f5920b72 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:09.395506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecc578-9a2c-4a0a-99fb-7c78f5920b72 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:09.395777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecc578-9a2c-4a0a-99fb-7c78f5920b72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:09.403620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-01ecc578-9a2c-4a0a-99fb-7c78f5920b72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:09.446917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ecc578-9a2c-4a0a-99fb-7c78f5920b72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:09.449556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ecc578-9a2c-4a0a-99fb-7c78f5920b72 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:09.475234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1160/2313] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:54:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:54:11.955005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418e539-791b-46ef-b0b6-a00720e97d41 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:11.955589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418e539-791b-46ef-b0b6-a00720e97d41 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:11.955589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418e539-791b-46ef-b0b6-a00720e97d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:11.980342 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-9418e539-791b-46ef-b0b6-a00720e97d41 None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:11.981668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9418e539-791b-46ef-b0b6-a00720e97d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:12.066281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1154/2314] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:54:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:12.074784 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecc578-9a2c-4a0a-99fb-7c78f5920b72 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:12.080280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-01ecc578-9a2c-4a0a-99fb-7c78f5920b72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:12.191047 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecc578-9a2c-4a0a-99fb-7c78f5920b72 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:12.204103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330a1ae-2cbb-432f-bafc-a74c434fd333 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:12.205113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330a1ae-2cbb-432f-bafc-a74c434fd333 service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:12.205401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330a1ae-2cbb-432f-bafc-a74c434fd333 service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:12.205675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330a1ae-2cbb-432f-bafc-a74c434fd333 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:12.216062 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330a1ae-2cbb-432f-bafc-a74c434fd333 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:12.217735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330a1ae-2cbb-432f-bafc-a74c434fd333 service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:12.229442 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1161/2315] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:54:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:54:22.053598 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=33 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:54:22.225079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=44 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:54:36.464177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049d31-5d56-499d-96f1-c77a4253dd04 None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:36.465355 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049d31-5d56-499d-96f1-c77a4253dd04 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:36.465792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049d31-5d56-499d-96f1-c77a4253dd04 None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:36.467673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1155/2316] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:54:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:54:36.486145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f88fa-f189-439e-8d00-20ba2ec58862 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:36.486941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f88fa-f189-439e-8d00-20ba2ec58862 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:36.487188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f88fa-f189-439e-8d00-20ba2ec58862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:36.534081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c83f88fa-f189-439e-8d00-20ba2ec58862 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:36.536402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c83f88fa-f189-439e-8d00-20ba2ec58862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:36.656371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1162/2317] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:54:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:36.669200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf049d31-5d56-499d-96f1-c77a4253dd04 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:36.689775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf049d31-5d56-499d-96f1-c77a4253dd04 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:36.693727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e64f8-8c6d-4d57-944f-aee4e361c3e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a59337d20e4678b8a16ac35bffbdb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:36.694695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e64f8-8c6d-4d57-944f-aee4e361c3e8 service glance] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:36.695213 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e64f8-8c6d-4d57-944f-aee4e361c3e8 service glance] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:36.695614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e64f8-8c6d-4d57-944f-aee4e361c3e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:36.709473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e64f8-8c6d-4d57-944f-aee4e361c3e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:36.711844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e64f8-8c6d-4d57-944f-aee4e361c3e8 service glance] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:36.728137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1156/2318] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:54:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:54:37.046406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf1c7b-7894-4797-bbdf-e6a2ef043b0b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:37.046738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf1c7b-7894-4797-bbdf-e6a2ef043b0b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:37.047017 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf1c7b-7894-4797-bbdf-e6a2ef043b0b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:37.048425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1163/2319] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:54:37.055903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c213f-cbfb-463c-ae5f-503ae89a2d27 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:37.056229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c213f-cbfb-463c-ae5f-503ae89a2d27 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:37.056509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c213f-cbfb-463c-ae5f-503ae89a2d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:37.083848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-662c213f-cbfb-463c-ae5f-503ae89a2d27 None None] MFA Rules not processed for user `863bcfa6b8694182b9d163133a141907`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:37.085678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-662c213f-cbfb-463c-ae5f-503ae89a2d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:37.148294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1157/2320] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:54:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:54:37.161259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf1c7b-7894-4797-bbdf-e6a2ef043b0b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:37.163886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-aabf1c7b-7894-4797-bbdf-e6a2ef043b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:37.178223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf1c7b-7894-4797-bbdf-e6a2ef043b0b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:37.182932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277227e-230b-4dca-8a22-1189f79cd712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:37.183807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277227e-230b-4dca-8a22-1189f79cd712 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:37.184072 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277227e-230b-4dca-8a22-1189f79cd712 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:37.184423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277227e-230b-4dca-8a22-1189f79cd712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:37.192769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277227e-230b-4dca-8a22-1189f79cd712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:37.195154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277227e-230b-4dca-8a22-1189f79cd712 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:37.206223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1164/2321] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:54:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:54:37.299100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb3342-7873-4c7b-b568-cbb5a974542c None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:37.299388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb3342-7873-4c7b-b568-cbb5a974542c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:37.299662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb3342-7873-4c7b-b568-cbb5a974542c None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:37.301230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1158/2322] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:54:37.308300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5319e-4384-49ab-9d98-577815a8e480 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:37.308711 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5319e-4384-49ab-9d98-577815a8e480 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:37.309246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5319e-4384-49ab-9d98-577815a8e480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:37.330826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3bc5319e-4384-49ab-9d98-577815a8e480 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:37.334405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3bc5319e-4384-49ab-9d98-577815a8e480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:37.342189 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1165/2323] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:54:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:37.356294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb3342-7873-4c7b-b568-cbb5a974542c None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:37.359138 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-fdcb3342-7873-4c7b-b568-cbb5a974542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:37.381059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb3342-7873-4c7b-b568-cbb5a974542c None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:37.387601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2aa2b-8ec7-48c3-95b7-d490159bbb64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:37.388482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2aa2b-8ec7-48c3-95b7-d490159bbb64 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:37.388758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2aa2b-8ec7-48c3-95b7-d490159bbb64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:37.389080 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2aa2b-8ec7-48c3-95b7-d490159bbb64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:37.401475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2aa2b-8ec7-48c3-95b7-d490159bbb64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:37.403048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2aa2b-8ec7-48c3-95b7-d490159bbb64 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:37.416765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1159/2324] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:54:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:54:43.697889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effec04-0bef-4acb-88d3-144ed8004639 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:43.698594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effec04-0bef-4acb-88d3-144ed8004639 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:43.698594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effec04-0bef-4acb-88d3-144ed8004639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:43.722772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9effec04-0bef-4acb-88d3-144ed8004639 None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:43.723927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9effec04-0bef-4acb-88d3-144ed8004639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:43.732477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1166/2325] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:54:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:43.756550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2aa2b-8ec7-48c3-95b7-d490159bbb64 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:43.768828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2aa2b-8ec7-48c3-95b7-d490159bbb64 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:43.774824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a43d5f-b848-4ed2-ba8c-1c1f732a49a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:43.775436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a43d5f-b848-4ed2-ba8c-1c1f732a49a0 service swift] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:43.775708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a43d5f-b848-4ed2-ba8c-1c1f732a49a0 service swift] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:43.775975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a43d5f-b848-4ed2-ba8c-1c1f732a49a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:43.785746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a43d5f-b848-4ed2-ba8c-1c1f732a49a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:43.787435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a43d5f-b848-4ed2-ba8c-1c1f732a49a0 service swift] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:43.797733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1160/2326] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:54:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:54:50.554985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effec04-0bef-4acb-88d3-144ed8004639 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:50.747893 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effec04-0bef-4acb-88d3-144ed8004639 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:50.754070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:50.754940 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:50.755320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:50.755793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] PATH_INFO: `/v3/users/cbb1d72d77ad4688bcbb567075ff556a` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:50.758859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb1d72d77ad4688bcbb567075ff556a)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:50.774618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:51.070320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb1d72d77ad4688bcbb567075ff556a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:51.105744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb1d72d77ad4688bcbb567075ff556a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:51.108889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:54:51.116278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb1d72d77ad4688bcbb567075ff556a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:51.421316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb1d72d77ad4688bcbb567075ff556a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:51.422595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb1d72d77ad4688bcbb567075ff556a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:51.439547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1167/2327] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:54:50 2022] DELETE /identity/v3/users/cbb1d72d77ad4688bcbb567075ff556a => generated 0 bytes in 880 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:54:51.440147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92612324-c4cb-4abd-8448-d7bff9d1ad00 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:51.440651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92612324-c4cb-4abd-8448-d7bff9d1ad00 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:51.440895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92612324-c4cb-4abd-8448-d7bff9d1ad00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:51.487636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-92612324-c4cb-4abd-8448-d7bff9d1ad00 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:51.489291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-92612324-c4cb-4abd-8448-d7bff9d1ad00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:51.621345 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1161/2328] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:51.637856 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:51.641466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:51.800150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c90fe7-575b-47cd-b431-2d2b0b82c86f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:51.808212 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273729ae-af98-439e-a3de-148f2dc6524e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:51.809675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273729ae-af98-439e-a3de-148f2dc6524e service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:51.810174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273729ae-af98-439e-a3de-148f2dc6524e service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:51.810734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273729ae-af98-439e-a3de-148f2dc6524e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:51.859756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273729ae-af98-439e-a3de-148f2dc6524e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:51.863399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273729ae-af98-439e-a3de-148f2dc6524e service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:51.907570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1168/2329] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:54:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:54:52.873498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92612324-c4cb-4abd-8448-d7bff9d1ad00 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:52.876860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-92612324-c4cb-4abd-8448-d7bff9d1ad00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:52.964663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92612324-c4cb-4abd-8448-d7bff9d1ad00 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:53.025104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:53.025104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:53.025818 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:53.028532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] PATH_INFO: `/v3/projects/9e8c274268854032aade285167d8080e` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:53.031487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e8c274268854032aade285167d8080e)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:53.051905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:53.334299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e8c274268854032aade285167d8080e', 'request_id': 'req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:53.495231 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e8c274268854032aade285167d8080e', 'request_id': 'req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:53.529599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e8c274268854032aade285167d8080e', 'request_id': 'req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:53.532571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:54:53.550505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] The token cache is being invalidate because project 9e8c274268854032aade285167d8080e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:54:53.550971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:53.551843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b31d3b5c-2cc3-46ba-b71f-c4e803c499c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:54:53.558111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1162/2330] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:54:52 2022] DELETE /identity/v3/projects/9e8c274268854032aade285167d8080e => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:54:53.568289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cfb31-b8fd-4509-8e42-c982a0e94f87 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:53.570974 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cfb31-b8fd-4509-8e42-c982a0e94f87 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:53.572056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cfb31-b8fd-4509-8e42-c982a0e94f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:53.621045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-064cfb31-b8fd-4509-8e42-c982a0e94f87 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:53.639365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-064cfb31-b8fd-4509-8e42-c982a0e94f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:53.806701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1169/2331] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:53.826495 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2b4b-36db-46c0-8302-32f0966332e2 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:53.826965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2b4b-36db-46c0-8302-32f0966332e2 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:53.827537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2b4b-36db-46c0-8302-32f0966332e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:53.901916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-0fee2b4b-36db-46c0-8302-32f0966332e2 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:53.906015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-0fee2b4b-36db-46c0-8302-32f0966332e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:53.931114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1163/2332] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:53.937213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cfb31-b8fd-4509-8e42-c982a0e94f87 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:53.964976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cfb31-b8fd-4509-8e42-c982a0e94f87 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:53.983689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92989ed4-fd76-4d24-9907-b4163d6b6f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:53.984386 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92989ed4-fd76-4d24-9907-b4163d6b6f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:53.984797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92989ed4-fd76-4d24-9907-b4163d6b6f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:53.984997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92989ed4-fd76-4d24-9907-b4163d6b6f8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:53.986349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92989ed4-fd76-4d24-9907-b4163d6b6f8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:54:53.986349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92989ed4-fd76-4d24-9907-b4163d6b6f8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:53.987606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92989ed4-fd76-4d24-9907-b4163d6b6f8d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:54.015552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1170/2333] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:54:54.025772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2b4b-36db-46c0-8302-32f0966332e2 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:54.075838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2b4b-36db-46c0-8302-32f0966332e2 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:54.116964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e2c7a-662c-456c-a24c-b9f1b5903cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:54.117774 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e2c7a-662c-456c-a24c-b9f1b5903cdc admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:54.118267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e2c7a-662c-456c-a24c-b9f1b5903cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:54.118592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e2c7a-662c-456c-a24c-b9f1b5903cdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:54.119343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e2c7a-662c-456c-a24c-b9f1b5903cdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:54.121354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e2c7a-662c-456c-a24c-b9f1b5903cdc admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:54.331955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1164/2334] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:54:54 2022] POST /identity/v3/projects => generated 375 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:54:54.338756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92989ed4-fd76-4d24-9907-b4163d6b6f8d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:54.353157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92989ed4-fd76-4d24-9907-b4163d6b6f8d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:54.496442 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc46f2-36df-4932-95a6-165eed1ee2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:54.497183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc46f2-36df-4932-95a6-165eed1ee2bd admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:54.497585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc46f2-36df-4932-95a6-165eed1ee2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:54.497957 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc46f2-36df-4932-95a6-165eed1ee2bd admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:54.498767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc46f2-36df-4932-95a6-165eed1ee2bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:54.500811 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc46f2-36df-4932-95a6-165eed1ee2bd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:54.614072 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1171/2335] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:54:54 2022] POST /identity/v3/users => generated 288 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:54:54.620600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e2c7a-662c-456c-a24c-b9f1b5903cdc admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:54.659964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e2c7a-662c-456c-a24c-b9f1b5903cdc admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:54.686734 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af6c8d-be16-4cd6-8673-393b28b3491f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:54.691001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af6c8d-be16-4cd6-8673-393b28b3491f admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:54.691294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af6c8d-be16-4cd6-8673-393b28b3491f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:54.691618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af6c8d-be16-4cd6-8673-393b28b3491f admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:54.692269 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af6c8d-be16-4cd6-8673-393b28b3491f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:54.694554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af6c8d-be16-4cd6-8673-393b28b3491f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:54.725222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1165/2336] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:54:54.729788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc46f2-36df-4932-95a6-165eed1ee2bd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:54.743174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:54:54.757438 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc46f2-36df-4932-95a6-165eed1ee2bd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:54.774997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07402a5b-7a68-4e05-9178-58e8daee4b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:54.775642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07402a5b-7a68-4e05-9178-58e8daee4b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:54.776065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07402a5b-7a68-4e05-9178-58e8daee4b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:54.776230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07402a5b-7a68-4e05-9178-58e8daee4b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:54.776843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07402a5b-7a68-4e05-9178-58e8daee4b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:54.781730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07402a5b-7a68-4e05-9178-58e8daee4b7c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:54.810497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1172/2337] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:54:54.815435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af6c8d-be16-4cd6-8673-393b28b3491f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:54.837532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af6c8d-be16-4cd6-8673-393b28b3491f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:54.857860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12746cb6-527e-4357-9854-4b5d7dbe89c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:54.862162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746cb6-527e-4357-9854-4b5d7dbe89c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:54.862162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746cb6-527e-4357-9854-4b5d7dbe89c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:54.862162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746cb6-527e-4357-9854-4b5d7dbe89c1 admin admin] PATH_INFO: `/v3/projects/044b897cfb074366854a48775f11c0da/users/a981c34be4924a8b9029589512ead5c5/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:54.907486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12746cb6-527e-4357-9854-4b5d7dbe89c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=044b897cfb074366854a48775f11c0da, user_id=a981c34be4924a8b9029589512ead5c5, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:54.910323 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12746cb6-527e-4357-9854-4b5d7dbe89c1 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:55.079557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1166/2338] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:54:54 2022] PUT /identity/v3/projects/044b897cfb074366854a48775f11c0da/users/a981c34be4924a8b9029589512ead5c5/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:54:55.096340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb3a97-02a9-4a94-8ea8-03713ec3c0aa None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:55.097054 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb3a97-02a9-4a94-8ea8-03713ec3c0aa None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:55.097553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb3a97-02a9-4a94-8ea8-03713ec3c0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:55.217472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-72bb3a97-02a9-4a94-8ea8-03713ec3c0aa None None] MFA Rules not processed for user `a981c34be4924a8b9029589512ead5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:55.219313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-72bb3a97-02a9-4a94-8ea8-03713ec3c0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:55.401840 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1173/2339] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:54:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:55.409906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12746cb6-527e-4357-9854-4b5d7dbe89c1 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:55.413859 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-12746cb6-527e-4357-9854-4b5d7dbe89c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:55.503670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12746cb6-527e-4357-9854-4b5d7dbe89c1 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:55.509327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6efa0-7481-4b5f-ae2b-875976f29f67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:55.509995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6efa0-7481-4b5f-ae2b-875976f29f67 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:55.510254 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6efa0-7481-4b5f-ae2b-875976f29f67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:55.512174 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6efa0-7481-4b5f-ae2b-875976f29f67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:55.524914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6efa0-7481-4b5f-ae2b-875976f29f67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:55.526718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6efa0-7481-4b5f-ae2b-875976f29f67 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:55.541222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1167/2340] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:54:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:54:57.949635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7a643-5818-41b6-a3a5-874e6f112efd None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:57.950287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7a643-5818-41b6-a3a5-874e6f112efd None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:57.951016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7a643-5818-41b6-a3a5-874e6f112efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:57.993588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c9b7a643-5818-41b6-a3a5-874e6f112efd None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:54:57.995864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c9b7a643-5818-41b6-a3a5-874e6f112efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:58.109943 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1174/2341] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:54:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:54:58.177015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6efa0-7481-4b5f-ae2b-875976f29f67 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:54:58.180313 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-39d6efa0-7481-4b5f-ae2b-875976f29f67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:54:58.396622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6efa0-7481-4b5f-ae2b-875976f29f67 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:54:58.413503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210faab-e72f-4efa-a598-8a1cd72a65ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:54:58.415319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210faab-e72f-4efa-a598-8a1cd72a65ad service swift] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:54:58.416417 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210faab-e72f-4efa-a598-8a1cd72a65ad service swift] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:54:58.417253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210faab-e72f-4efa-a598-8a1cd72a65ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:54:58.439403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210faab-e72f-4efa-a598-8a1cd72a65ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:54:58.443784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210faab-e72f-4efa-a598-8a1cd72a65ad service swift] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:54:58.468988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1168/2342] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:54:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:55:06.608907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7a643-5818-41b6-a3a5-874e6f112efd None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:06.610346 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c9b7a643-5818-41b6-a3a5-874e6f112efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:06.733447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7a643-5818-41b6-a3a5-874e6f112efd None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:06.748724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d4831-4781-4bb8-b81b-e7c27bd1689a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:06.751217 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d4831-4781-4bb8-b81b-e7c27bd1689a service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:06.751514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d4831-4781-4bb8-b81b-e7c27bd1689a service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:06.751791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d4831-4781-4bb8-b81b-e7c27bd1689a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:06.755791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-596d4831-4781-4bb8-b81b-e7c27bd1689a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:06.909176 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:55:07.039629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d4831-4781-4bb8-b81b-e7c27bd1689a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:07.041360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d4831-4781-4bb8-b81b-e7c27bd1689a service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:07.057642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1175/2343] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:55:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:55:08.455010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=88,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:55:08.491125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f345da-b92a-4d82-8c21-0c04ee960bda None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:08.491402 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f345da-b92a-4d82-8c21-0c04ee960bda None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:08.491665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f345da-b92a-4d82-8c21-0c04ee960bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:08.542183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d2f345da-b92a-4d82-8c21-0c04ee960bda None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:55:08.544315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d2f345da-b92a-4d82-8c21-0c04ee960bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:08.556610 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1169/2344] 10.222.0.58 () {58 vars in 967 bytes} [Mon Aug 29 23:55:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:55:08.617507 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d4831-4781-4bb8-b81b-e7c27bd1689a service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:08.657043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d4831-4781-4bb8-b81b-e7c27bd1689a service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:08.678614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d92ad5-a1bf-402d-b671-e62835a83d85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:08.679445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d92ad5-a1bf-402d-b671-e62835a83d85 service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:08.679799 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d92ad5-a1bf-402d-b671-e62835a83d85 service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:08.680188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d92ad5-a1bf-402d-b671-e62835a83d85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:08.716063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d92ad5-a1bf-402d-b671-e62835a83d85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:08.718681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d92ad5-a1bf-402d-b671-e62835a83d85 service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:08.753337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1176/2345] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:55:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:55:13.123332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f345da-b92a-4d82-8c21-0c04ee960bda None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:13.124809 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d2f345da-b92a-4d82-8c21-0c04ee960bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:13.283493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f345da-b92a-4d82-8c21-0c04ee960bda None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:13.290049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:13.291096 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:13.291909 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:13.292538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] PATH_INFO: `/v3/users/74d57012e9a04960b6c5dde2243dafbc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:13.294062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d57012e9a04960b6c5dde2243dafbc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:13.296379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:13.575548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d57012e9a04960b6c5dde2243dafbc'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:13.597007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d57012e9a04960b6c5dde2243dafbc'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:13.615623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d57012e9a04960b6c5dde2243dafbc'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:13.768772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d57012e9a04960b6c5dde2243dafbc'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:13.769621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d57012e9a04960b6c5dde2243dafbc'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:13.784678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1170/2346] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:55:13 2022] DELETE /identity/v3/users/74d57012e9a04960b6c5dde2243dafbc => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:55:14.245546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d92ad5-a1bf-402d-b671-e62835a83d85 service swift] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:14.267039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d92ad5-a1bf-402d-b671-e62835a83d85 service swift] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:14.288576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:14.292742 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:14.483998 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5040e-368b-4c44-9987-822ab6afca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:14.485740 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5040e-368b-4c44-9987-822ab6afca24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:14.492186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5040e-368b-4c44-9987-822ab6afca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:14.493553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5040e-368b-4c44-9987-822ab6afca24 admin admin] PATH_INFO: `/v3/users/a981c34be4924a8b9029589512ead5c5` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:14.501549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5040e-368b-4c44-9987-822ab6afca24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a981c34be4924a8b9029589512ead5c5)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:14.505812 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5040e-368b-4c44-9987-822ab6afca24 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:14.575807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186e38a-65c0-41c5-b563-dcf61b8aa553 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:14.581558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:14.582606 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:14.583215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:14.583660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] PATH_INFO: `/v3/projects/87299c60c42b43e7bf7e8f92f30ae87d` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:14.585656 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=87299c60c42b43e7bf7e8f92f30ae87d)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:14.587912 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:14.612952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:55:14.780642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87299c60c42b43e7bf7e8f92f30ae87d', 'request_id': 'req-b388295e-b81d-41c4-aef7-67e0887efd0e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:14.862163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87299c60c42b43e7bf7e8f92f30ae87d', 'request_id': 'req-b388295e-b81d-41c4-aef7-67e0887efd0e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:14.883818 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5cc5040e-368b-4c44-9987-822ab6afca24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a981c34be4924a8b9029589512ead5c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:14.888697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87299c60c42b43e7bf7e8f92f30ae87d', 'request_id': 'req-b388295e-b81d-41c4-aef7-67e0887efd0e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:14.909498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5cc5040e-368b-4c44-9987-822ab6afca24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a981c34be4924a8b9029589512ead5c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:14.941589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] The token cache is being invalidate because project 87299c60c42b43e7bf7e8f92f30ae87d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:55:14.941884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:14.945051 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b388295e-b81d-41c4-aef7-67e0887efd0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:14.947009 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5cc5040e-368b-4c44-9987-822ab6afca24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a981c34be4924a8b9029589512ead5c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:14.955372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1171/2347] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:55:14 2022] DELETE /identity/v3/projects/87299c60c42b43e7bf7e8f92f30ae87d => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:55:14.957054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbbe05-c2c6-44fd-abb9-516b886646f2 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:14.957336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbbe05-c2c6-44fd-abb9-516b886646f2 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:14.957609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbbe05-c2c6-44fd-abb9-516b886646f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:15.003867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d1cbbe05-c2c6-44fd-abb9-516b886646f2 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:55:15.005253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d1cbbe05-c2c6-44fd-abb9-516b886646f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:15.046230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5cc5040e-368b-4c44-9987-822ab6afca24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a981c34be4924a8b9029589512ead5c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:15.048888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5cc5040e-368b-4c44-9987-822ab6afca24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a981c34be4924a8b9029589512ead5c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:15.072025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1177/2348] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:55:14 2022] DELETE /identity/v3/users/a981c34be4924a8b9029589512ead5c5 => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:55:15.079792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188748b-46a7-4e9f-b8d5-f51f763f1790 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:15.084134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188748b-46a7-4e9f-b8d5-f51f763f1790 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:15.085272 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188748b-46a7-4e9f-b8d5-f51f763f1790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:15.157108 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9188748b-46a7-4e9f-b8d5-f51f763f1790 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:55:15.159286 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9188748b-46a7-4e9f-b8d5-f51f763f1790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:15.198546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1172/2349] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:55:15.212279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162970a6-3034-4278-a22c-cd752e057f65 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:15.212833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162970a6-3034-4278-a22c-cd752e057f65 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:15.213271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162970a6-3034-4278-a22c-cd752e057f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:15.245781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-162970a6-3034-4278-a22c-cd752e057f65 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:55:15.247130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-162970a6-3034-4278-a22c-cd752e057f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:15.276919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:55:15.350751 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1178/2350] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:55:15.362808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1173/2351] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:55:15.363425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188748b-46a7-4e9f-b8d5-f51f763f1790 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:15.364224 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162970a6-3034-4278-a22c-cd752e057f65 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:15.366372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9188748b-46a7-4e9f-b8d5-f51f763f1790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:15.388866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162970a6-3034-4278-a22c-cd752e057f65 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:15.393069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c673f-3c1d-4f66-9568-6f85c417441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:15.393730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c673f-3c1d-4f66-9568-6f85c417441a admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:15.394011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c673f-3c1d-4f66-9568-6f85c417441a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:15.394286 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c673f-3c1d-4f66-9568-6f85c417441a admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:15.395171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c673f-3c1d-4f66-9568-6f85c417441a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:55:15.395484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c673f-3c1d-4f66-9568-6f85c417441a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:15.398263 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c673f-3c1d-4f66-9568-6f85c417441a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:15.413700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1174/2352] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:55:15.418637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c673f-3c1d-4f66-9568-6f85c417441a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:15.432729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c673f-3c1d-4f66-9568-6f85c417441a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:15.438530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ebb35-0dad-469e-a028-c855d8664a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:15.439460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ebb35-0dad-469e-a028-c855d8664a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:15.439903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ebb35-0dad-469e-a028-c855d8664a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:15.440358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ebb35-0dad-469e-a028-c855d8664a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:15.441369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ebb35-0dad-469e-a028-c855d8664a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:15.444278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ebb35-0dad-469e-a028-c855d8664a72 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:15.480659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188748b-46a7-4e9f-b8d5-f51f763f1790 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:15.487020 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeba6f-bc2c-42bb-836d-49d7c13b1efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:15.488209 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeba6f-bc2c-42bb-836d-49d7c13b1efb service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:15.488769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeba6f-bc2c-42bb-836d-49d7c13b1efb service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:15.490145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeba6f-bc2c-42bb-836d-49d7c13b1efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:15.502299 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeba6f-bc2c-42bb-836d-49d7c13b1efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:15.503931 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeba6f-bc2c-42bb-836d-49d7c13b1efb service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:15.515921 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1179/2353] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:55:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:55:15.563911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1175/2354] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:55:15 2022] POST /identity/v3/projects => generated 371 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:55:15.567860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeba6f-bc2c-42bb-836d-49d7c13b1efb service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:15.594067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeba6f-bc2c-42bb-836d-49d7c13b1efb service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:15.711500 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af073d8f-a9d5-4af9-ae06-225f8756d2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:15.712297 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af073d8f-a9d5-4af9-ae06-225f8756d2ef admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:15.712646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af073d8f-a9d5-4af9-ae06-225f8756d2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:15.712967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af073d8f-a9d5-4af9-ae06-225f8756d2ef admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:15.713783 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af073d8f-a9d5-4af9-ae06-225f8756d2ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:15.716662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af073d8f-a9d5-4af9-ae06-225f8756d2ef admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:15.845028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1180/2355] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:55:15 2022] POST /identity/v3/users => generated 286 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:55:15.850838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ebb35-0dad-469e-a028-c855d8664a72 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:15.875339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ebb35-0dad-469e-a028-c855d8664a72 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:15.882337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c31e7-1012-4ebf-b26f-243600f8712e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:15.883167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c31e7-1012-4ebf-b26f-243600f8712e admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:15.883456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c31e7-1012-4ebf-b26f-243600f8712e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:15.883733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c31e7-1012-4ebf-b26f-243600f8712e admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:15.884472 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c31e7-1012-4ebf-b26f-243600f8712e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:15.906700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c31e7-1012-4ebf-b26f-243600f8712e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:15.927193 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:55:15.961010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af073d8f-a9d5-4af9-ae06-225f8756d2ef admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:15.974863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1176/2356] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:55:15.983696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af073d8f-a9d5-4af9-ae06-225f8756d2ef admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:16.008259 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848debe-23fd-4bed-b150-a1036f16574d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:16.018536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848debe-23fd-4bed-b150-a1036f16574d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:16.019792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848debe-23fd-4bed-b150-a1036f16574d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:16.021156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848debe-23fd-4bed-b150-a1036f16574d admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:16.028776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848debe-23fd-4bed-b150-a1036f16574d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:16.032976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848debe-23fd-4bed-b150-a1036f16574d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:16.058141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1181/2357] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:55:16.065220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c31e7-1012-4ebf-b26f-243600f8712e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:16.078582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c31e7-1012-4ebf-b26f-243600f8712e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:16.113877 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6691500-f107-4ee3-b2ed-b95927de6cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:16.115807 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6691500-f107-4ee3-b2ed-b95927de6cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:16.116268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6691500-f107-4ee3-b2ed-b95927de6cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:16.117757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6691500-f107-4ee3-b2ed-b95927de6cbe admin admin] PATH_INFO: `/v3/projects/81ef8dbbbbb94c2ba634fffae53e8dc3/users/fa2219a29f7c45b1bad2a0606660e238/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:16.149609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6691500-f107-4ee3-b2ed-b95927de6cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ef8dbbbbb94c2ba634fffae53e8dc3, user_id=fa2219a29f7c45b1bad2a0606660e238, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:16.152752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6691500-f107-4ee3-b2ed-b95927de6cbe admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:16.268177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1177/2358] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:55:16 2022] PUT /identity/v3/projects/81ef8dbbbbb94c2ba634fffae53e8dc3/users/fa2219a29f7c45b1bad2a0606660e238/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:55:16.291182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0795e-d178-4134-a2e5-c64e54328bd9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:16.292586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0795e-d178-4134-a2e5-c64e54328bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:16.295090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0795e-d178-4134-a2e5-c64e54328bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:16.396630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6691500-f107-4ee3-b2ed-b95927de6cbe admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:16.400185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e6691500-f107-4ee3-b2ed-b95927de6cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:16.432089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f7f0795e-d178-4134-a2e5-c64e54328bd9 None None] MFA Rules not processed for user `fa2219a29f7c45b1bad2a0606660e238`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:55:16.447184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f7f0795e-d178-4134-a2e5-c64e54328bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:16.686173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6691500-f107-4ee3-b2ed-b95927de6cbe admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:16.702164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:16.703330 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:16.703627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:16.703962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] PATH_INFO: `/v3/projects/044b897cfb074366854a48775f11c0da` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:16.706270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:55:16.707563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=044b897cfb074366854a48775f11c0da)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:16.723892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:16.744856 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1182/2359] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:55:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:55:16.772769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0795e-d178-4134-a2e5-c64e54328bd9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:16.775525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f7f0795e-d178-4134-a2e5-c64e54328bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:16.940672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '044b897cfb074366854a48775f11c0da', 'request_id': 'req-a83a1281-e9fe-4d5d-a548-2400502dd8a3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:16.972897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0795e-d178-4134-a2e5-c64e54328bd9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:17.026943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '044b897cfb074366854a48775f11c0da', 'request_id': 'req-a83a1281-e9fe-4d5d-a548-2400502dd8a3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:17.043889 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '044b897cfb074366854a48775f11c0da', 'request_id': 'req-a83a1281-e9fe-4d5d-a548-2400502dd8a3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:17.060834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] The token cache is being invalidate because project 044b897cfb074366854a48775f11c0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:55:17.061391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:17.062858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a83a1281-e9fe-4d5d-a548-2400502dd8a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:17.064932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1178/2360] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:55:16 2022] DELETE /identity/v3/projects/044b897cfb074366854a48775f11c0da => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:55:17.138628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cea325-d46b-40bd-ba66-7eb221bf1919 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a59337d20e4678b8a16ac35bffbdb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:17.142457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cea325-d46b-40bd-ba66-7eb221bf1919 service glance] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:17.142457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cea325-d46b-40bd-ba66-7eb221bf1919 service glance] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:17.142457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cea325-d46b-40bd-ba66-7eb221bf1919 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:17.156607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-53cea325-d46b-40bd-ba66-7eb221bf1919 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:17.387572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cea325-d46b-40bd-ba66-7eb221bf1919 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:17.389083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cea325-d46b-40bd-ba66-7eb221bf1919 service glance] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:17.418593 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1183/2361] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:55:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:55:20.119273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69889c1f-42d2-44c8-b6a2-2c00e1826219 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:20.119870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69889c1f-42d2-44c8-b6a2-2c00e1826219 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:20.119870 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69889c1f-42d2-44c8-b6a2-2c00e1826219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:20.147614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-69889c1f-42d2-44c8-b6a2-2c00e1826219 None None] MFA Rules not processed for user `734d44438bfb4a79a68aa6705e49b1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:55:20.148876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-69889c1f-42d2-44c8-b6a2-2c00e1826219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:20.235763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1179/2362] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:55:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:55:27.416903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=70 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:55:29.771795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4a79-153a-488f-8e91-9dfdd44560b9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:29.772413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4a79-153a-488f-8e91-9dfdd44560b9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:29.772413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4a79-153a-488f-8e91-9dfdd44560b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:29.804358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-629f4a79-153a-488f-8e91-9dfdd44560b9 None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:55:29.805610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-629f4a79-153a-488f-8e91-9dfdd44560b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:29.876517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1184/2363] 10.222.0.58 () {58 vars in 967 bytes} [Mon Aug 29 23:55:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:55:29.895795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69889c1f-42d2-44c8-b6a2-2c00e1826219 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:29.898379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-69889c1f-42d2-44c8-b6a2-2c00e1826219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:29.916971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:55:30.043218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69889c1f-42d2-44c8-b6a2-2c00e1826219 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:30.052702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7433a7-9c8e-40a5-8fd4-fc8126bc5dd1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:30.053544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7433a7-9c8e-40a5-8fd4-fc8126bc5dd1 service swift] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:30.053960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7433a7-9c8e-40a5-8fd4-fc8126bc5dd1 service swift] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:30.054393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7433a7-9c8e-40a5-8fd4-fc8126bc5dd1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:30.065411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7433a7-9c8e-40a5-8fd4-fc8126bc5dd1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:30.068939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7433a7-9c8e-40a5-8fd4-fc8126bc5dd1 service swift] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:30.084842 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1180/2364] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:55:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 23:55:36.827813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31a03c-f0e9-4f9c-a44c-7219be9d33b7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:36.828365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31a03c-f0e9-4f9c-a44c-7219be9d33b7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:36.828365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31a03c-f0e9-4f9c-a44c-7219be9d33b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:36.853562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3b31a03c-f0e9-4f9c-a44c-7219be9d33b7 None None] MFA Rules not processed for user `34ef7ee27c3948ac9df1ff94ce24618e`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:55:36.854911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3b31a03c-f0e9-4f9c-a44c-7219be9d33b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:36.927070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1185/2365] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:55:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:55:40.076852 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=28 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:55:46.916786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=50 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:55:49.575108 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7433a7-9c8e-40a5-8fd4-fc8126bc5dd1 service swift] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:49.576462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bf7433a7-9c8e-40a5-8fd4-fc8126bc5dd1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:49.647733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7433a7-9c8e-40a5-8fd4-fc8126bc5dd1 service swift] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:49.652209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:49.652828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:49.653105 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:49.653375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] PATH_INFO: `/v3/users/fa2219a29f7c45b1bad2a0606660e238` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:49.654519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa2219a29f7c45b1bad2a0606660e238)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:49.656447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:49.877270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa2219a29f7c45b1bad2a0606660e238'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:49.893205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa2219a29f7c45b1bad2a0606660e238'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:49.901192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa2219a29f7c45b1bad2a0606660e238'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:49.979098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa2219a29f7c45b1bad2a0606660e238'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:49.979723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa2219a29f7c45b1bad2a0606660e238'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:49.986197 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1181/2366] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:55:49 2022] DELETE /identity/v3/users/fa2219a29f7c45b1bad2a0606660e238 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:55:49.990778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37023ff0-b673-495c-a7b8-bd93447b9b64 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:49.991271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37023ff0-b673-495c-a7b8-bd93447b9b64 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:49.991371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37023ff0-b673-495c-a7b8-bd93447b9b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:50.012212 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-37023ff0-b673-495c-a7b8-bd93447b9b64 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:55:50.013422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-37023ff0-b673-495c-a7b8-bd93447b9b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:50.073430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1186/2367] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:55:50.085217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:50.086998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:50.157479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c4410-0e31-4004-a898-c79a41343381 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:50.161148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a78412-a7de-4710-b341-944eca4f0b89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:50.161892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a78412-a7de-4710-b341-944eca4f0b89 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:50.162170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a78412-a7de-4710-b341-944eca4f0b89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:50.162461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a78412-a7de-4710-b341-944eca4f0b89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:50.172717 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a78412-a7de-4710-b341-944eca4f0b89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:50.174194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a78412-a7de-4710-b341-944eca4f0b89 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:50.186008 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1182/2368] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:55:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:55:50.730164 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37023ff0-b673-495c-a7b8-bd93447b9b64 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:55:50.731848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-37023ff0-b673-495c-a7b8-bd93447b9b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:50.749193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37023ff0-b673-495c-a7b8-bd93447b9b64 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:55:50.754311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:55:50.755598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:50.755888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:50.756149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] PATH_INFO: `/v3/projects/81ef8dbbbbb94c2ba634fffae53e8dc3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:50.757334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ef8dbbbbb94c2ba634fffae53e8dc3)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:55:50.759306 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:55:50.896676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ef8dbbbbb94c2ba634fffae53e8dc3', 'request_id': 'req-cd45b790-bc64-4446-a7bd-032c98c9e0c2'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:50.902363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ef8dbbbbb94c2ba634fffae53e8dc3', 'request_id': 'req-cd45b790-bc64-4446-a7bd-032c98c9e0c2'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:50.978878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ef8dbbbbb94c2ba634fffae53e8dc3', 'request_id': 'req-cd45b790-bc64-4446-a7bd-032c98c9e0c2'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:51.000467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] The token cache is being invalidate because project 81ef8dbbbbb94c2ba634fffae53e8dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:55:51.001167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:51.002024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:55:51.003933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1187/2369] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:55:50 2022] DELETE /identity/v3/projects/81ef8dbbbbb94c2ba634fffae53e8dc3 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:55:58.897205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2a570-02c7-4e35-9d43-0e28818cf4e5 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:55:58.897745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2a570-02c7-4e35-9d43-0e28818cf4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:55:58.897880 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2a570-02c7-4e35-9d43-0e28818cf4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:55:58.918701 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-21b2a570-02c7-4e35-9d43-0e28818cf4e5 None None] MFA Rules not processed for user `db604b7af623452fac9b3712b6978168`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:55:58.920062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-21b2a570-02c7-4e35-9d43-0e28818cf4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:55:58.938168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:55:58.978250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1183/2370] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:55:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:56:00.997233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:03.080689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:03.081990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:03.152374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45b790-bc64-4446-a7bd-032c98c9e0c2 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:03.157243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873951ce-3ec7-4457-ae18-5c5175d65376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:03.157908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873951ce-3ec7-4457-ae18-5c5175d65376 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:03.158220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873951ce-3ec7-4457-ae18-5c5175d65376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:03.158762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873951ce-3ec7-4457-ae18-5c5175d65376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:03.161905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-873951ce-3ec7-4457-ae18-5c5175d65376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:03.253551 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873951ce-3ec7-4457-ae18-5c5175d65376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:03.255830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873951ce-3ec7-4457-ae18-5c5175d65376 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:03.268333 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1188/2371] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:05.784239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2a570-02c7-4e35-9d43-0e28818cf4e5 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:05.794101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2a570-02c7-4e35-9d43-0e28818cf4e5 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:05.797614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfd1eb-a144-4507-bba1-ee7d66f34daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:05.798208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfd1eb-a144-4507-bba1-ee7d66f34daa service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:05.798516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfd1eb-a144-4507-bba1-ee7d66f34daa service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:05.798791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfd1eb-a144-4507-bba1-ee7d66f34daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:05.800514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-31cfd1eb-a144-4507-bba1-ee7d66f34daa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:05.814117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfd1eb-a144-4507-bba1-ee7d66f34daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:05.815593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfd1eb-a144-4507-bba1-ee7d66f34daa service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:05.828125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1184/2372] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:12.985693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873951ce-3ec7-4457-ae18-5c5175d65376 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:12.987270 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-873951ce-3ec7-4457-ae18-5c5175d65376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:13.000686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873951ce-3ec7-4457-ae18-5c5175d65376 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:13.007194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f18466-61cd-42a9-8cad-40f05c2a174c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:13.008074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f18466-61cd-42a9-8cad-40f05c2a174c service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:13.008863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f18466-61cd-42a9-8cad-40f05c2a174c service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:13.009401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f18466-61cd-42a9-8cad-40f05c2a174c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:13.011279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e5f18466-61cd-42a9-8cad-40f05c2a174c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:13.028418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f18466-61cd-42a9-8cad-40f05c2a174c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:13.032072 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f18466-61cd-42a9-8cad-40f05c2a174c service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:13.045056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1189/2373] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:15.823357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=21 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:20.667214 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfd1eb-a144-4507-bba1-ee7d66f34daa service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:20.668727 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-31cfd1eb-a144-4507-bba1-ee7d66f34daa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:20.683691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfd1eb-a144-4507-bba1-ee7d66f34daa service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:20.690018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bde00-38c0-428b-ba3f-2758cf81bb1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:20.690722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bde00-38c0-428b-ba3f-2758cf81bb1a service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:20.691152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bde00-38c0-428b-ba3f-2758cf81bb1a service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:20.691656 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bde00-38c0-428b-ba3f-2758cf81bb1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:20.693720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-fc3bde00-38c0-428b-ba3f-2758cf81bb1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:20.770887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bde00-38c0-428b-ba3f-2758cf81bb1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:20.774646 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bde00-38c0-428b-ba3f-2758cf81bb1a service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:20.793947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1185/2374] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:23.038734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=60 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:26.496781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb3236-d02b-44b6-850d-cf4094b67382 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:26.497876 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb3236-d02b-44b6-850d-cf4094b67382 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:26.497876 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb3236-d02b-44b6-850d-cf4094b67382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:26.520990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8ecb3236-d02b-44b6-850d-cf4094b67382 None None] MFA Rules not processed for user `90b2c6a74c29443b94d1577b39c7ce0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:26.522241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8ecb3236-d02b-44b6-850d-cf4094b67382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:26.582265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1190/2375] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:56:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:56:26.596166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bde00-38c0-428b-ba3f-2758cf81bb1a service nova] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:26.609299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bde00-38c0-428b-ba3f-2758cf81bb1a service nova] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:26.615058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745df2fc-488d-44f5-96b8-97e0d67b5ea4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:26.615676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745df2fc-488d-44f5-96b8-97e0d67b5ea4 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:26.616023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745df2fc-488d-44f5-96b8-97e0d67b5ea4 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:26.616899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745df2fc-488d-44f5-96b8-97e0d67b5ea4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:26.618779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-745df2fc-488d-44f5-96b8-97e0d67b5ea4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:26.630520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745df2fc-488d-44f5-96b8-97e0d67b5ea4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:26.632148 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745df2fc-488d-44f5-96b8-97e0d67b5ea4 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:26.642646 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1186/2376] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:56:26.979754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb3236-d02b-44b6-850d-cf4094b67382 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:26.989684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb3236-d02b-44b6-850d-cf4094b67382 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:26.993843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7c9f6-5ffc-46a6-8940-bcea51b4ab34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:26.994493 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7c9f6-5ffc-46a6-8940-bcea51b4ab34 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:26.994771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7c9f6-5ffc-46a6-8940-bcea51b4ab34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:26.995042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7c9f6-5ffc-46a6-8940-bcea51b4ab34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:26.996807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-90e7c9f6-5ffc-46a6-8940-bcea51b4ab34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:27.008081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7c9f6-5ffc-46a6-8940-bcea51b4ab34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:27.009681 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7c9f6-5ffc-46a6-8940-bcea51b4ab34 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:27.020392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1191/2377] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:56:27.476495 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745df2fc-488d-44f5-96b8-97e0d67b5ea4 service nova] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:27.478797 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-745df2fc-488d-44f5-96b8-97e0d67b5ea4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:27.598520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745df2fc-488d-44f5-96b8-97e0d67b5ea4 service nova] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:27.612015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b591772-8fa2-4f05-bbe6-babb66fee182 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:27.619731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b591772-8fa2-4f05-bbe6-babb66fee182 service placement] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:27.619731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b591772-8fa2-4f05-bbe6-babb66fee182 service placement] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:27.619731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b591772-8fa2-4f05-bbe6-babb66fee182 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:27.621466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5b591772-8fa2-4f05-bbe6-babb66fee182 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:27.639434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b591772-8fa2-4f05-bbe6-babb66fee182 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:27.642867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b591772-8fa2-4f05-bbe6-babb66fee182 service placement] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:27.661032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1187/2378] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:37.015021 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=31 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:37.653940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=66 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:38.768415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7c9f6-5ffc-46a6-8940-bcea51b4ab34 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:38.770302 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-90e7c9f6-5ffc-46a6-8940-bcea51b4ab34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:38.799090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7c9f6-5ffc-46a6-8940-bcea51b4ab34 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:38.799090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de3e38-1c98-4e5f-aff3-7f1895e00653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:38.799090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de3e38-1c98-4e5f-aff3-7f1895e00653 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:38.799090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de3e38-1c98-4e5f-aff3-7f1895e00653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:38.799380 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de3e38-1c98-4e5f-aff3-7f1895e00653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:38.799380 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a4de3e38-1c98-4e5f-aff3-7f1895e00653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:38.859590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de3e38-1c98-4e5f-aff3-7f1895e00653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:38.861065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de3e38-1c98-4e5f-aff3-7f1895e00653 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:38.871653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1192/2379] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:40.210167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b591772-8fa2-4f05-bbe6-babb66fee182 service placement] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:40.226085 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b591772-8fa2-4f05-bbe6-babb66fee182 service placement] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:40.232821 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61335a0c-1bc5-4e56-a67a-c7228852a8f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:40.233588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61335a0c-1bc5-4e56-a67a-c7228852a8f3 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:40.233988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61335a0c-1bc5-4e56-a67a-c7228852a8f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:40.234132 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61335a0c-1bc5-4e56-a67a-c7228852a8f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:40.239079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-61335a0c-1bc5-4e56-a67a-c7228852a8f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:40.304335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61335a0c-1bc5-4e56-a67a-c7228852a8f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:40.305957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61335a0c-1bc5-4e56-a67a-c7228852a8f3 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:40.315514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1188/2380] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:43.298602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468fc5c-3fd2-4ee8-bdd2-9317c19625bc None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:43.298602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468fc5c-3fd2-4ee8-bdd2-9317c19625bc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:43.298602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468fc5c-3fd2-4ee8-bdd2-9317c19625bc None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:43.298602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1193/2381] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:56:43.312386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732378a9-6179-4343-90f9-43f9ef6674b9 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:43.313267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732378a9-6179-4343-90f9-43f9ef6674b9 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:43.313613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732378a9-6179-4343-90f9-43f9ef6674b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:43.344393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-732378a9-6179-4343-90f9-43f9ef6674b9 None None] MFA Rules not processed for user `fb770d4379e74bf79ad4280329c771c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:43.346019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-732378a9-6179-4343-90f9-43f9ef6674b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:43.357626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1189/2382] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:56:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:56:43.386084 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468fc5c-3fd2-4ee8-bdd2-9317c19625bc None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:43.388958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e468fc5c-3fd2-4ee8-bdd2-9317c19625bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:43.420532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468fc5c-3fd2-4ee8-bdd2-9317c19625bc None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:43.437025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952890f5-243d-4ed0-b1bb-25991da05c98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:43.437964 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952890f5-243d-4ed0-b1bb-25991da05c98 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:43.438401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952890f5-243d-4ed0-b1bb-25991da05c98 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:43.439081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952890f5-243d-4ed0-b1bb-25991da05c98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:43.458167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952890f5-243d-4ed0-b1bb-25991da05c98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:43.463424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952890f5-243d-4ed0-b1bb-25991da05c98 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:43.488195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1194/2383] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:43.520268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732378a9-6179-4343-90f9-43f9ef6674b9 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:43.522253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-732378a9-6179-4343-90f9-43f9ef6674b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:43.545981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732378a9-6179-4343-90f9-43f9ef6674b9 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:43.555220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d5279-02fa-4240-9ca6-443aef9e3747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:43.557049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d5279-02fa-4240-9ca6-443aef9e3747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:43.557567 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d5279-02fa-4240-9ca6-443aef9e3747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:43.557955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d5279-02fa-4240-9ca6-443aef9e3747 admin admin] PATH_INFO: `/v3/users/734d44438bfb4a79a68aa6705e49b1e9` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:43.559591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d5279-02fa-4240-9ca6-443aef9e3747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=734d44438bfb4a79a68aa6705e49b1e9)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:43.562551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d5279-02fa-4240-9ca6-443aef9e3747 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:43.867883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-086d5279-02fa-4240-9ca6-443aef9e3747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '734d44438bfb4a79a68aa6705e49b1e9'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:43.882691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-086d5279-02fa-4240-9ca6-443aef9e3747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '734d44438bfb4a79a68aa6705e49b1e9'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:43.891291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-086d5279-02fa-4240-9ca6-443aef9e3747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '734d44438bfb4a79a68aa6705e49b1e9'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:43.983922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-086d5279-02fa-4240-9ca6-443aef9e3747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '734d44438bfb4a79a68aa6705e49b1e9'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:43.987899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-086d5279-02fa-4240-9ca6-443aef9e3747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '734d44438bfb4a79a68aa6705e49b1e9'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:43.992894 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1190/2384] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:56:43 2022] DELETE /identity/v3/users/734d44438bfb4a79a68aa6705e49b1e9 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:44.505945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952890f5-243d-4ed0-b1bb-25991da05c98 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:44.519015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952890f5-243d-4ed0-b1bb-25991da05c98 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:44.598022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:44.599033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:44.599358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:44.599640 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] PATH_INFO: `/v3/projects/cb38bc3ae5c14e4c8385809d84573600` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:44.602095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb38bc3ae5c14e4c8385809d84573600)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:44.604167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:44.831518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb38bc3ae5c14e4c8385809d84573600', 'request_id': 'req-f3dc298b-d869-4259-9b63-4e0cdbda59ac'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:44.838126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb38bc3ae5c14e4c8385809d84573600', 'request_id': 'req-f3dc298b-d869-4259-9b63-4e0cdbda59ac'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:44.944273 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb38bc3ae5c14e4c8385809d84573600', 'request_id': 'req-f3dc298b-d869-4259-9b63-4e0cdbda59ac'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:44.968657 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] The token cache is being invalidate because project cb38bc3ae5c14e4c8385809d84573600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:56:44.969219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:44.969979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f3dc298b-d869-4259-9b63-4e0cdbda59ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:44.982828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1195/2385] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:56:44 2022] DELETE /identity/v3/projects/cb38bc3ae5c14e4c8385809d84573600 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:44.984346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b167e80-ba12-45a8-ad2f-360ec2b84981 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:44.984998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b167e80-ba12-45a8-ad2f-360ec2b84981 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:44.984998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b167e80-ba12-45a8-ad2f-360ec2b84981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:45.010517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-6b167e80-ba12-45a8-ad2f-360ec2b84981 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:45.011573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6b167e80-ba12-45a8-ad2f-360ec2b84981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:45.095334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1191/2386] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:56:45.103619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2941b-3f6a-4cd0-9928-b1b0274e6c7e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:45.103905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2941b-3f6a-4cd0-9928-b1b0274e6c7e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:45.104171 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2941b-3f6a-4cd0-9928-b1b0274e6c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:45.127067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b1b2941b-3f6a-4cd0-9928-b1b0274e6c7e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:45.128321 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b1b2941b-3f6a-4cd0-9928-b1b0274e6c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:45.136441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1196/2387] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:56:45.140851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b167e80-ba12-45a8-ad2f-360ec2b84981 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:45.149306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:45.154188 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b167e80-ba12-45a8-ad2f-360ec2b84981 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:45.165941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e250d3-0511-48a5-b14d-688d80231ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:45.165941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e250d3-0511-48a5-b14d-688d80231ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:45.165941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e250d3-0511-48a5-b14d-688d80231ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:45.165941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e250d3-0511-48a5-b14d-688d80231ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:45.166706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e250d3-0511-48a5-b14d-688d80231ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:56:45.166706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e250d3-0511-48a5-b14d-688d80231ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:45.166706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e250d3-0511-48a5-b14d-688d80231ea3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:45.177366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1192/2388] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:56:45.181847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2941b-3f6a-4cd0-9928-b1b0274e6c7e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:45.187800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:45.191980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2941b-3f6a-4cd0-9928-b1b0274e6c7e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:45.198030 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceabd5-69cb-4a36-ae91-b51cba0f365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:45.198767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceabd5-69cb-4a36-ae91-b51cba0f365a admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:45.199122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceabd5-69cb-4a36-ae91-b51cba0f365a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:45.199406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceabd5-69cb-4a36-ae91-b51cba0f365a admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:45.200052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ceabd5-69cb-4a36-ae91-b51cba0f365a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:45.202016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ceabd5-69cb-4a36-ae91-b51cba0f365a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:45.295598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1197/2389] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:56:45 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:56:45.301246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e250d3-0511-48a5-b14d-688d80231ea3 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:45.322010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e250d3-0511-48a5-b14d-688d80231ea3 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:45.423091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a5ac1-d227-4df0-a3a3-cc3ac9cf42e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:45.424145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a5ac1-d227-4df0-a3a3-cc3ac9cf42e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:45.424643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a5ac1-d227-4df0-a3a3-cc3ac9cf42e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:45.425133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a5ac1-d227-4df0-a3a3-cc3ac9cf42e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:45.426300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a5ac1-d227-4df0-a3a3-cc3ac9cf42e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:45.430591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a5ac1-d227-4df0-a3a3-cc3ac9cf42e8 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:45.588971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1193/2390] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:56:45 2022] POST /identity/v3/users => generated 286 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:56:45.592960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceabd5-69cb-4a36-ae91-b51cba0f365a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:45.605809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceabd5-69cb-4a36-ae91-b51cba0f365a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:45.610909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c3c08-5f80-41cf-9447-685f0b6c48d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:45.611590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c3c08-5f80-41cf-9447-685f0b6c48d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:45.611954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c3c08-5f80-41cf-9447-685f0b6c48d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:45.612411 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c3c08-5f80-41cf-9447-685f0b6c48d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:45.613224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c3c08-5f80-41cf-9447-685f0b6c48d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:45.615013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c3c08-5f80-41cf-9447-685f0b6c48d5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:45.635826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1198/2391] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:56:45.639678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a5ac1-d227-4df0-a3a3-cc3ac9cf42e8 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:45.653143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a5ac1-d227-4df0-a3a3-cc3ac9cf42e8 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:45.658356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab047b49-c7d4-4983-8b70-4689a5828aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:45.659221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab047b49-c7d4-4983-8b70-4689a5828aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:45.659505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab047b49-c7d4-4983-8b70-4689a5828aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:45.659776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab047b49-c7d4-4983-8b70-4689a5828aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:45.660438 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab047b49-c7d4-4983-8b70-4689a5828aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:45.662162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab047b49-c7d4-4983-8b70-4689a5828aaf admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:45.679774 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1194/2392] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:56:45.684044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c3c08-5f80-41cf-9447-685f0b6c48d5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:45.696335 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c3c08-5f80-41cf-9447-685f0b6c48d5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:45.701344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7be929-1f5f-4edc-a529-d1d160352682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:45.702028 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7be929-1f5f-4edc-a529-d1d160352682 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:45.702306 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7be929-1f5f-4edc-a529-d1d160352682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:45.707976 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7be929-1f5f-4edc-a529-d1d160352682 admin admin] PATH_INFO: `/v3/projects/1b0e837d11f9456fa947f14dd0499947/users/809a613eab10447ca8a8cf3aa87d9820/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:45.721879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7be929-1f5f-4edc-a529-d1d160352682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b0e837d11f9456fa947f14dd0499947, user_id=809a613eab10447ca8a8cf3aa87d9820, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:45.724775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7be929-1f5f-4edc-a529-d1d160352682 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:45.870515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1199/2393] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:56:45 2022] PUT /identity/v3/projects/1b0e837d11f9456fa947f14dd0499947/users/809a613eab10447ca8a8cf3aa87d9820/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:45.883381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f12977-8dbd-4d15-b892-0faa525df4e7 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:45.883961 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f12977-8dbd-4d15-b892-0faa525df4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:45.884447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f12977-8dbd-4d15-b892-0faa525df4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:45.932032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-23f12977-8dbd-4d15-b892-0faa525df4e7 None None] MFA Rules not processed for user `809a613eab10447ca8a8cf3aa87d9820`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:45.933692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-23f12977-8dbd-4d15-b892-0faa525df4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:46.022772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1195/2394] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:56:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:56:46.034593 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7be929-1f5f-4edc-a529-d1d160352682 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:46.039076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6f7be929-1f5f-4edc-a529-d1d160352682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:46.147311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7be929-1f5f-4edc-a529-d1d160352682 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:46.153992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375f7cb-7d12-4b63-abf5-850ef8b21382 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:46.154842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375f7cb-7d12-4b63-abf5-850ef8b21382 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:46.155231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375f7cb-7d12-4b63-abf5-850ef8b21382 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:46.155602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375f7cb-7d12-4b63-abf5-850ef8b21382 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:46.164722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375f7cb-7d12-4b63-abf5-850ef8b21382 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:46.166477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375f7cb-7d12-4b63-abf5-850ef8b21382 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:46.189956 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1200/2395] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:46.217690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9030-3582-47f8-94b3-e8abe19fa426 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:46.217947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9030-3582-47f8-94b3-e8abe19fa426 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:46.218325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9030-3582-47f8-94b3-e8abe19fa426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:46.251200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-248e9030-3582-47f8-94b3-e8abe19fa426 None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:46.252147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-248e9030-3582-47f8-94b3-e8abe19fa426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:46.342146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1196/2396] 10.222.0.58 () {58 vars in 967 bytes} [Mon Aug 29 23:56:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:56:46.520278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375f7cb-7d12-4b63-abf5-850ef8b21382 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:46.520278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d375f7cb-7d12-4b63-abf5-850ef8b21382 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:46.606885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375f7cb-7d12-4b63-abf5-850ef8b21382 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:46.615961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd201500-189c-4032-b028-abbea45599cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:46.615961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd201500-189c-4032-b028-abbea45599cd service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:46.615961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd201500-189c-4032-b028-abbea45599cd service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:46.615961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd201500-189c-4032-b028-abbea45599cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:46.625006 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd201500-189c-4032-b028-abbea45599cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:46.626930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd201500-189c-4032-b028-abbea45599cd service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:46.637118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1201/2397] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:56:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:56:49.738861 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e9030-3582-47f8-94b3-e8abe19fa426 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:49.743844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:49.809595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e9030-3582-47f8-94b3-e8abe19fa426 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:49.814716 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:49.815634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:49.816070 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:49.816476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] PATH_INFO: `/v3/users/809a613eab10447ca8a8cf3aa87d9820` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:49.817769 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=809a613eab10447ca8a8cf3aa87d9820)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:49.819867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:50.081785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '809a613eab10447ca8a8cf3aa87d9820'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:50.106245 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '809a613eab10447ca8a8cf3aa87d9820'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:50.116826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '809a613eab10447ca8a8cf3aa87d9820'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:50.183829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '809a613eab10447ca8a8cf3aa87d9820'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:50.184607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '809a613eab10447ca8a8cf3aa87d9820'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:50.193346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1197/2398] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:56:49 2022] DELETE /identity/v3/users/809a613eab10447ca8a8cf3aa87d9820 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:50.200041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f68164-faff-4c65-995f-2ecaba17351d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:50.200864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f68164-faff-4c65-995f-2ecaba17351d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:50.200864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f68164-faff-4c65-995f-2ecaba17351d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:50.231646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c7f68164-faff-4c65-995f-2ecaba17351d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:50.234340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c7f68164-faff-4c65-995f-2ecaba17351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:50.274219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:50.326097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1202/2399] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:56:50.337154 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:50.338761 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:50.439962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9134bd-7502-4d68-b903-541a1b315669 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:50.443916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e009f0-480c-457f-97e7-cfd747efd288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:50.445490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e009f0-480c-457f-97e7-cfd747efd288 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:50.445843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e009f0-480c-457f-97e7-cfd747efd288 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:50.446212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e009f0-480c-457f-97e7-cfd747efd288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:50.458885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e009f0-480c-457f-97e7-cfd747efd288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:50.461396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e009f0-480c-457f-97e7-cfd747efd288 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:50.473149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1198/2400] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:51.580833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f68164-faff-4c65-995f-2ecaba17351d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:51.585937 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c7f68164-faff-4c65-995f-2ecaba17351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:51.614735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f68164-faff-4c65-995f-2ecaba17351d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:51.624650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:51.626167 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:51.626981 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:51.627977 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] PATH_INFO: `/v3/projects/1b0e837d11f9456fa947f14dd0499947` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:51.630295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b0e837d11f9456fa947f14dd0499947)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:51.633720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:51.827529 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b0e837d11f9456fa947f14dd0499947', 'request_id': 'req-f79c18bf-1b54-4371-aade-7acc0b4ce65e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:51.841201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b0e837d11f9456fa947f14dd0499947', 'request_id': 'req-f79c18bf-1b54-4371-aade-7acc0b4ce65e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:51.921475 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b0e837d11f9456fa947f14dd0499947', 'request_id': 'req-f79c18bf-1b54-4371-aade-7acc0b4ce65e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:51.933802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] The token cache is being invalidate because project 1b0e837d11f9456fa947f14dd0499947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:56:51.934098 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:51.934823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f79c18bf-1b54-4371-aade-7acc0b4ce65e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:51.936339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1203/2401] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:56:51 2022] DELETE /identity/v3/projects/1b0e837d11f9456fa947f14dd0499947 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:51.941825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735d555-f2b0-4208-9ab9-ef7342a4c3db None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:51.942352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735d555-f2b0-4208-9ab9-ef7342a4c3db None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:51.942352 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735d555-f2b0-4208-9ab9-ef7342a4c3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:51.966353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d735d555-f2b0-4208-9ab9-ef7342a4c3db None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:51.968163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d735d555-f2b0-4208-9ab9-ef7342a4c3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:51.990676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:52.085483 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1199/2402] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:56:52.100959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce2058-d74c-4234-93c0-e7d6e137c838 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:52.101460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce2058-d74c-4234-93c0-e7d6e137c838 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:52.102301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce2058-d74c-4234-93c0-e7d6e137c838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:52.170252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-69ce2058-d74c-4234-93c0-e7d6e137c838 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:52.174203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-69ce2058-d74c-4234-93c0-e7d6e137c838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:52.198034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1204/2403] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:56:52.202506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735d555-f2b0-4208-9ab9-ef7342a4c3db None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:52.213566 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735d555-f2b0-4208-9ab9-ef7342a4c3db None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:52.219315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff101329-d9ec-4cf0-a78c-d9c27c7bbf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:52.221735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101329-d9ec-4cf0-a78c-d9c27c7bbf65 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:52.221735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101329-d9ec-4cf0-a78c-d9c27c7bbf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:52.221735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101329-d9ec-4cf0-a78c-d9c27c7bbf65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:52.221735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff101329-d9ec-4cf0-a78c-d9c27c7bbf65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:56:52.222499 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff101329-d9ec-4cf0-a78c-d9c27c7bbf65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:52.224579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff101329-d9ec-4cf0-a78c-d9c27c7bbf65 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:52.240159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1200/2404] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:56:52.247986 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce2058-d74c-4234-93c0-e7d6e137c838 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:52.264762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce2058-d74c-4234-93c0-e7d6e137c838 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:52.279406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0d313-841b-49de-83aa-18019f303ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:52.283532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0d313-841b-49de-83aa-18019f303ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:52.284709 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0d313-841b-49de-83aa-18019f303ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:52.285219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0d313-841b-49de-83aa-18019f303ad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:52.286741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0d313-841b-49de-83aa-18019f303ad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:52.291350 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0d313-841b-49de-83aa-18019f303ad4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:52.495156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1205/2405] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:56:52 2022] POST /identity/v3/projects => generated 381 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:56:52.512292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff101329-d9ec-4cf0-a78c-d9c27c7bbf65 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:52.536294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff101329-d9ec-4cf0-a78c-d9c27c7bbf65 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:52.679888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65de80-639b-48eb-8af7-95490e6e13f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:52.680848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65de80-639b-48eb-8af7-95490e6e13f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:52.681344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65de80-639b-48eb-8af7-95490e6e13f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:52.681773 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65de80-639b-48eb-8af7-95490e6e13f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:52.682640 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65de80-639b-48eb-8af7-95490e6e13f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:52.684885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65de80-639b-48eb-8af7-95490e6e13f8 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:52.814910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1201/2406] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:56:52 2022] POST /identity/v3/users => generated 291 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:56:52.818952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0d313-841b-49de-83aa-18019f303ad4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:52.835676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0d313-841b-49de-83aa-18019f303ad4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:52.844097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b656799-6bac-43e5-a0c7-feddbe3ef6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:52.845160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b656799-6bac-43e5-a0c7-feddbe3ef6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:52.845604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b656799-6bac-43e5-a0c7-feddbe3ef6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:52.846052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b656799-6bac-43e5-a0c7-feddbe3ef6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:52.846984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b656799-6bac-43e5-a0c7-feddbe3ef6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:52.848877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b656799-6bac-43e5-a0c7-feddbe3ef6c5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:52.861501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1206/2407] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:56:52.867405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65de80-639b-48eb-8af7-95490e6e13f8 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:52.879772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65de80-639b-48eb-8af7-95490e6e13f8 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:52.888134 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6b19f-2bd1-4291-a370-b67bff98f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:52.888544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6b19f-2bd1-4291-a370-b67bff98f0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:52.888866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6b19f-2bd1-4291-a370-b67bff98f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:52.889093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6b19f-2bd1-4291-a370-b67bff98f0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:52.889771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6b19f-2bd1-4291-a370-b67bff98f0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:52.891484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6b19f-2bd1-4291-a370-b67bff98f0c3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:52.910811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1202/2408] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:56:52.913460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b656799-6bac-43e5-a0c7-feddbe3ef6c5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:52.932592 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b656799-6bac-43e5-a0c7-feddbe3ef6c5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:52.939484 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1994539-db66-4f6e-af4e-9b1fea7f7af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:52.941268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1994539-db66-4f6e-af4e-9b1fea7f7af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:52.942040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1994539-db66-4f6e-af4e-9b1fea7f7af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:52.942833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1994539-db66-4f6e-af4e-9b1fea7f7af6 admin admin] PATH_INFO: `/v3/projects/3ab7770082504a10ba4e3981b9d62d8e/users/4326be512fed45fe8d8290a71163d7fc/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:52.988410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1994539-db66-4f6e-af4e-9b1fea7f7af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7770082504a10ba4e3981b9d62d8e, user_id=4326be512fed45fe8d8290a71163d7fc, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:52.992604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1994539-db66-4f6e-af4e-9b1fea7f7af6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:53.111253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1207/2409] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:56:52 2022] PUT /identity/v3/projects/3ab7770082504a10ba4e3981b9d62d8e/users/4326be512fed45fe8d8290a71163d7fc/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:56:53.124805 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db9542-4f2f-46af-8d97-4cc85fd21eec None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:53.125365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db9542-4f2f-46af-8d97-4cc85fd21eec None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:53.125550 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db9542-4f2f-46af-8d97-4cc85fd21eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:53.160904 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d2db9542-4f2f-46af-8d97-4cc85fd21eec None None] MFA Rules not processed for user `4326be512fed45fe8d8290a71163d7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:53.162315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d2db9542-4f2f-46af-8d97-4cc85fd21eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:53.267964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1203/2410] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:56:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:56:53.274207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1994539-db66-4f6e-af4e-9b1fea7f7af6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:53.359447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:53.380485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1994539-db66-4f6e-af4e-9b1fea7f7af6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:53.385814 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb9300-cb16-496a-b79a-d0c796eb3c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:53.386533 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb9300-cb16-496a-b79a-d0c796eb3c38 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:53.386822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb9300-cb16-496a-b79a-d0c796eb3c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:53.387102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb9300-cb16-496a-b79a-d0c796eb3c38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:53.387865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb9300-cb16-496a-b79a-d0c796eb3c38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:53.390026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb9300-cb16-496a-b79a-d0c796eb3c38 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:53.523569 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1208/2411] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:56:53 2022] POST /identity/v3/projects => generated 381 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:56:53.538760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db9542-4f2f-46af-8d97-4cc85fd21eec None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:53.558293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db9542-4f2f-46af-8d97-4cc85fd21eec None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:53.592984 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:53.652076 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c7ed6-926e-4bcd-b90f-d7e49790899e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:53.652703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c7ed6-926e-4bcd-b90f-d7e49790899e admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:53.653012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c7ed6-926e-4bcd-b90f-d7e49790899e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:53.653275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c7ed6-926e-4bcd-b90f-d7e49790899e admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:53.653907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c7ed6-926e-4bcd-b90f-d7e49790899e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:53.655998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c7ed6-926e-4bcd-b90f-d7e49790899e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:53.790503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1204/2412] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:56:53 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:56:53.794585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb9300-cb16-496a-b79a-d0c796eb3c38 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:53.806108 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb9300-cb16-496a-b79a-d0c796eb3c38 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:53.811566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaf7c4-f7e4-4dfc-8e7d-7c447ee83dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:53.812184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaf7c4-f7e4-4dfc-8e7d-7c447ee83dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:53.812465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaf7c4-f7e4-4dfc-8e7d-7c447ee83dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:53.812739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaf7c4-f7e4-4dfc-8e7d-7c447ee83dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:53.813408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aaf7c4-f7e4-4dfc-8e7d-7c447ee83dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:53.815921 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aaf7c4-f7e4-4dfc-8e7d-7c447ee83dc0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:53.828410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1209/2413] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:56:53.848681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c7ed6-926e-4bcd-b90f-d7e49790899e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:53.860035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c7ed6-926e-4bcd-b90f-d7e49790899e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:53.864801 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad056a83-c857-4e05-b17b-3e9cadf491b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:53.865500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad056a83-c857-4e05-b17b-3e9cadf491b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:53.865845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad056a83-c857-4e05-b17b-3e9cadf491b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:53.866178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad056a83-c857-4e05-b17b-3e9cadf491b7 admin admin] PATH_INFO: `/v3/projects/859ace971b1d427ba987371a90db9587/users/badc80aba84c45fd937fd5998b6d69aa/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:53.883152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad056a83-c857-4e05-b17b-3e9cadf491b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=859ace971b1d427ba987371a90db9587, user_id=badc80aba84c45fd937fd5998b6d69aa, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:53.888049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad056a83-c857-4e05-b17b-3e9cadf491b7 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:53.984720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1205/2414] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:56:53 2022] PUT /identity/v3/projects/859ace971b1d427ba987371a90db9587/users/badc80aba84c45fd937fd5998b6d69aa/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:53.989222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaf7c4-f7e4-4dfc-8e7d-7c447ee83dc0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:54.001914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaf7c4-f7e4-4dfc-8e7d-7c447ee83dc0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:54.102142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91230fe9-b8d2-4b45-8446-a93c399e6c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:54.103806 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91230fe9-b8d2-4b45-8446-a93c399e6c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:54.104307 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91230fe9-b8d2-4b45-8446-a93c399e6c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:54.105203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91230fe9-b8d2-4b45-8446-a93c399e6c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:54.107937 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91230fe9-b8d2-4b45-8446-a93c399e6c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:54.110486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91230fe9-b8d2-4b45-8446-a93c399e6c4e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:54.125802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1210/2415] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:56:54.129742 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad056a83-c857-4e05-b17b-3e9cadf491b7 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:54.144066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad056a83-c857-4e05-b17b-3e9cadf491b7 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:54.155356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627489a-03d3-4a5b-a729-a26b85c4383d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:54.156165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627489a-03d3-4a5b-a729-a26b85c4383d admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:54.159572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627489a-03d3-4a5b-a729-a26b85c4383d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:54.159572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627489a-03d3-4a5b-a729-a26b85c4383d admin admin] PATH_INFO: `/v3/projects/859ace971b1d427ba987371a90db9587/users/badc80aba84c45fd937fd5998b6d69aa/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:54.159572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627489a-03d3-4a5b-a729-a26b85c4383d admin admin] RBAC: Authorizing `identity:create_grant(project_id=859ace971b1d427ba987371a90db9587, user_id=badc80aba84c45fd937fd5998b6d69aa, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:54.162710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627489a-03d3-4a5b-a729-a26b85c4383d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:54.186487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1206/2416] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:56:54 2022] PUT /identity/v3/projects/859ace971b1d427ba987371a90db9587/users/badc80aba84c45fd937fd5998b6d69aa/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:54.195734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339cd18-248c-49f4-b705-216722d13a8f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:54.196137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339cd18-248c-49f4-b705-216722d13a8f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:54.196720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339cd18-248c-49f4-b705-216722d13a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:54.230623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a339cd18-248c-49f4-b705-216722d13a8f None None] MFA Rules not processed for user `badc80aba84c45fd937fd5998b6d69aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:54.232059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a339cd18-248c-49f4-b705-216722d13a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:54.346582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1211/2417] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:56:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:56:54.359427 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627489a-03d3-4a5b-a729-a26b85c4383d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:54.361182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3627489a-03d3-4a5b-a729-a26b85c4383d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:54.420925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627489a-03d3-4a5b-a729-a26b85c4383d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:54.424458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee1344-8ece-422e-b756-545734965205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:54.425063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee1344-8ece-422e-b756-545734965205 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:54.425336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee1344-8ece-422e-b756-545734965205 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:54.425606 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee1344-8ece-422e-b756-545734965205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:54.436758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee1344-8ece-422e-b756-545734965205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:54.438237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee1344-8ece-422e-b756-545734965205 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:54.450748 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1207/2418] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:55.649547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339cd18-248c-49f4-b705-216722d13a8f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:55.659972 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339cd18-248c-49f4-b705-216722d13a8f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:55.666072 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2f82a-79ab-47a1-9228-0af9627c4839 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'badc80aba84c45fd937fd5998b6d69aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '859ace971b1d427ba987371a90db9587', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:55.666775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2f82a-79ab-47a1-9228-0af9627c4839 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:55.667045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2f82a-79ab-47a1-9228-0af9627c4839 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:55.667336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2f82a-79ab-47a1-9228-0af9627c4839 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:55.668140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2f82a-79ab-47a1-9228-0af9627c4839 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:56:55.668454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2f82a-79ab-47a1-9228-0af9627c4839 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:55.670149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2f82a-79ab-47a1-9228-0af9627c4839 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:55.687983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1212/2419] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:56:55.694232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee1344-8ece-422e-b756-545734965205 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:55.704677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee1344-8ece-422e-b756-545734965205 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:55.711089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47706fd-4553-4f8a-8ebd-12283cba7769 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'badc80aba84c45fd937fd5998b6d69aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '859ace971b1d427ba987371a90db9587', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:55.711991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47706fd-4553-4f8a-8ebd-12283cba7769 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:55.712349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47706fd-4553-4f8a-8ebd-12283cba7769 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:55.712694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47706fd-4553-4f8a-8ebd-12283cba7769 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:55.713422 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47706fd-4553-4f8a-8ebd-12283cba7769 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:55.715569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47706fd-4553-4f8a-8ebd-12283cba7769 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:55.837936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1208/2420] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:56:55 2022] POST /identity/v3/projects => generated 354 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:56:55.842394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2f82a-79ab-47a1-9228-0af9627c4839 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:55.853357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2f82a-79ab-47a1-9228-0af9627c4839 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:55.875599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:55.931087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e17e5-8b1e-4ab4-94f3-f77d7246eae1 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'badc80aba84c45fd937fd5998b6d69aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '859ace971b1d427ba987371a90db9587', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:55.931087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e17e5-8b1e-4ab4-94f3-f77d7246eae1 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:55.931087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e17e5-8b1e-4ab4-94f3-f77d7246eae1 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:55.931987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e17e5-8b1e-4ab4-94f3-f77d7246eae1 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:55.931987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e17e5-8b1e-4ab4-94f3-f77d7246eae1 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:56:55.931987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e17e5-8b1e-4ab4-94f3-f77d7246eae1 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:55.931987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e17e5-8b1e-4ab4-94f3-f77d7246eae1 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:55.952120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1213/2421] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:56:56.003836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47706fd-4553-4f8a-8ebd-12283cba7769 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:56.025705 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47706fd-4553-4f8a-8ebd-12283cba7769 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:56.033900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'badc80aba84c45fd937fd5998b6d69aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '859ace971b1d427ba987371a90db9587', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:56.035017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:56.035527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:56.036052 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] PATH_INFO: `/v3/projects/21458d9bc5144c3096f7eba1f545d5e9` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:56.038901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: Authorizing `identity:delete_project(project_id=21458d9bc5144c3096f7eba1f545d5e9)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:56.042766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:56.268684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21458d9bc5144c3096f7eba1f545d5e9', 'request_id': 'req-e36c6856-c10f-4cf2-911f-812c6ac3c995'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:56.270938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:56.355483 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21458d9bc5144c3096f7eba1f545d5e9', 'request_id': 'req-e36c6856-c10f-4cf2-911f-812c6ac3c995'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:56.363200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21458d9bc5144c3096f7eba1f545d5e9', 'request_id': 'req-e36c6856-c10f-4cf2-911f-812c6ac3c995'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:56.378653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] The token cache is being invalidate because project 21458d9bc5144c3096f7eba1f545d5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:56:56.379096 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:56.380682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:56.383022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1209/2422] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:56:56 2022] DELETE /identity/v3/projects/21458d9bc5144c3096f7eba1f545d5e9 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:57.195990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e17e5-8b1e-4ab4-94f3-f77d7246eae1 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:57.198658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a36e17e5-8b1e-4ab4-94f3-f77d7246eae1 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:57.288907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e17e5-8b1e-4ab4-94f3-f77d7246eae1 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:57.386906 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb247d-7bef-4fd6-af7b-56a2a2267207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:57.387730 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb247d-7bef-4fd6-af7b-56a2a2267207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:57.388311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb247d-7bef-4fd6-af7b-56a2a2267207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:57.388694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb247d-7bef-4fd6-af7b-56a2a2267207 admin admin] PATH_INFO: `/v3/users/4326be512fed45fe8d8290a71163d7fc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:57.390173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb247d-7bef-4fd6-af7b-56a2a2267207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4326be512fed45fe8d8290a71163d7fc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:57.392252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb247d-7bef-4fd6-af7b-56a2a2267207 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:57.559583 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:57.561128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:57.697022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a2fb247d-7bef-4fd6-af7b-56a2a2267207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4326be512fed45fe8d8290a71163d7fc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:57.715542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a2fb247d-7bef-4fd6-af7b-56a2a2267207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4326be512fed45fe8d8290a71163d7fc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:57.718148 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:57.719920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c6856-c10f-4cf2-911f-812c6ac3c995 tempest-VolumeQuotaClassesTest-1372701172 tempest-VolumeQuotaClassesTest-1372701172-project-admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:57.729694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a2fb247d-7bef-4fd6-af7b-56a2a2267207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4326be512fed45fe8d8290a71163d7fc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:57.732429 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8108a-0182-43e3-936c-165c3388f898 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:57.733218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8108a-0182-43e3-936c-165c3388f898 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:57.733558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8108a-0182-43e3-936c-165c3388f898 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:57.734228 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8108a-0182-43e3-936c-165c3388f898 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:57.736761 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1fd8108a-0182-43e3-936c-165c3388f898 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:57.756822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8108a-0182-43e3-936c-165c3388f898 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:57.758998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8108a-0182-43e3-936c-165c3388f898 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:57.770974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1210/2423] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:56:57.772694 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8108a-0182-43e3-936c-165c3388f898 service nova] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:57.774412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1fd8108a-0182-43e3-936c-165c3388f898 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:57.779419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a2fb247d-7bef-4fd6-af7b-56a2a2267207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4326be512fed45fe8d8290a71163d7fc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:57.780486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a2fb247d-7bef-4fd6-af7b-56a2a2267207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4326be512fed45fe8d8290a71163d7fc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:57.790310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1214/2424] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:56:57 2022] DELETE /identity/v3/users/4326be512fed45fe8d8290a71163d7fc => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:57.794325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8108a-0182-43e3-936c-165c3388f898 service nova] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:57.796762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155670a1-3235-4df9-b17f-b936e3523612 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:57.798847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155670a1-3235-4df9-b17f-b936e3523612 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:57.799292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155670a1-3235-4df9-b17f-b936e3523612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:57.830905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-155670a1-3235-4df9-b17f-b936e3523612 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:56:57.832275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-155670a1-3235-4df9-b17f-b936e3523612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:57.889366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:57.889979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:57.890260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:57.890576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] PATH_INFO: `/v3/users/27f78b50d76341e08e184582e8400a04` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:57.891893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27f78b50d76341e08e184582e8400a04)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:57.893923 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:57.929357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1215/2425] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:56:57.941268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155670a1-3235-4df9-b17f-b936e3523612 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:57.943330 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-155670a1-3235-4df9-b17f-b936e3523612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:58.029120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155670a1-3235-4df9-b17f-b936e3523612 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:58.039033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e798-e025-4ade-bd10-3dfba3584e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:58.039813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e798-e025-4ade-bd10-3dfba3584e8b service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:58.040169 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e798-e025-4ade-bd10-3dfba3584e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:58.040444 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e798-e025-4ade-bd10-3dfba3584e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:58.053239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900e798-e025-4ade-bd10-3dfba3584e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:58.056290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900e798-e025-4ade-bd10-3dfba3584e8b service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:58.066496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1216/2426] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:56:58.136848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27f78b50d76341e08e184582e8400a04'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:58.154198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27f78b50d76341e08e184582e8400a04'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:58.155939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:INSERT=1,keystone:DELETE=12,keystone:SELECT=87 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:58.163941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27f78b50d76341e08e184582e8400a04'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:58.338461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27f78b50d76341e08e184582e8400a04'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:58.339110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27f78b50d76341e08e184582e8400a04'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:58.351235 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1211/2427] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:56:57 2022] DELETE /identity/v3/users/27f78b50d76341e08e184582e8400a04 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:58.844424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e798-e025-4ade-bd10-3dfba3584e8b service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:58.847672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7900e798-e025-4ade-bd10-3dfba3584e8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:58.851829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:58.853627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:58.871696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e798-e025-4ade-bd10-3dfba3584e8b service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:58.952644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:58.953281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:58.953558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:58.953828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] PATH_INFO: `/v3/projects/3ab7770082504a10ba4e3981b9d62d8e` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:58.955004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ab7770082504a10ba4e3981b9d62d8e)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:58.956688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3485db-829b-45de-be1f-b749b58f1768 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:58.956925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:58.962108 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:58.962790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:58.963073 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:58.963348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] PATH_INFO: `/v3/projects/2f665e85ae5b4008b60765fd3076a331` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:58.965013 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f665e85ae5b4008b60765fd3076a331)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:58.966467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:59.128150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:59.146332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f665e85ae5b4008b60765fd3076a331', 'request_id': 'req-8e4d3a85-6311-4b38-9098-e5c597131fdc'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.148611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ab7770082504a10ba4e3981b9d62d8e', 'request_id': 'req-7e1b4777-6513-46ca-83d4-373181e7fb88'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.156425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ab7770082504a10ba4e3981b9d62d8e', 'request_id': 'req-7e1b4777-6513-46ca-83d4-373181e7fb88'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.229917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f665e85ae5b4008b60765fd3076a331', 'request_id': 'req-8e4d3a85-6311-4b38-9098-e5c597131fdc'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.237267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f665e85ae5b4008b60765fd3076a331', 'request_id': 'req-8e4d3a85-6311-4b38-9098-e5c597131fdc'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.254642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] The token cache is being invalidate because project 2f665e85ae5b4008b60765fd3076a331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:56:59.254934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.255597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.257794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1212/2428] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:56:58 2022] DELETE /identity/v3/projects/2f665e85ae5b4008b60765fd3076a331 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:59.262643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:59.264034 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:59.315378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ab7770082504a10ba4e3981b9d62d8e', 'request_id': 'req-7e1b4777-6513-46ca-83d4-373181e7fb88'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.328517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] The token cache is being invalidate because project 3ab7770082504a10ba4e3981b9d62d8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:56:59.328805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.329444 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.330994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1217/2429] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:56:58 2022] DELETE /identity/v3/projects/3ab7770082504a10ba4e3981b9d62d8e => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:56:59.331521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d3a85-6311-4b38-9098-e5c597131fdc admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:59.335857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:59.337876 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:56:59.338704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:59.339364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:59.339642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:59.339913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] PATH_INFO: `/v3/users/31df9500e6d3467c9e944be3c368e77b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:59.341151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] RBAC: Authorizing `identity:delete_user(user_id=31df9500e6d3467c9e944be3c368e77b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:59.343251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:59.358864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:56:59.406184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4777-6513-46ca-83d4-373181e7fb88 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:56:59.410877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:56:59.411506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:56:59.411782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:56:59.412056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] PATH_INFO: `/v3/users/badc80aba84c45fd937fd5998b6d69aa` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:56:59.414152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=badc80aba84c45fd937fd5998b6d69aa)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:56:59.416351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:56:59.518682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31df9500e6d3467c9e944be3c368e77b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.532881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31df9500e6d3467c9e944be3c368e77b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.540068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31df9500e6d3467c9e944be3c368e77b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.642348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31df9500e6d3467c9e944be3c368e77b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.645825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31df9500e6d3467c9e944be3c368e77b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.654461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1213/2430] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:56:59 2022] DELETE /identity/v3/users/31df9500e6d3467c9e944be3c368e77b => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:56:59.660449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'badc80aba84c45fd937fd5998b6d69aa'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.674889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'badc80aba84c45fd937fd5998b6d69aa'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.683237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'badc80aba84c45fd937fd5998b6d69aa'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.778324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'badc80aba84c45fd937fd5998b6d69aa'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.780545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'badc80aba84c45fd937fd5998b6d69aa'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:56:59.788609 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1218/2431] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:56:59 2022] DELETE /identity/v3/users/badc80aba84c45fd937fd5998b6d69aa => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:56:59.980170 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:56:59.981622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:00.054063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7c28b-5cb6-4286-a244-3a878215944a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:00.143252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:00.143252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:00.143525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:00.143775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] PATH_INFO: `/v3/projects/c487dbdcde1f4ef098cd8bab7a1b7e27` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:00.144841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c487dbdcde1f4ef098cd8bab7a1b7e27)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:00.146872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:00.292636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:00.306552 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c487dbdcde1f4ef098cd8bab7a1b7e27', 'request_id': 'req-8095fdf2-04a5-4527-ad81-7809ac20d270'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:00.400970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c487dbdcde1f4ef098cd8bab7a1b7e27', 'request_id': 'req-8095fdf2-04a5-4527-ad81-7809ac20d270'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:00.408978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c487dbdcde1f4ef098cd8bab7a1b7e27', 'request_id': 'req-8095fdf2-04a5-4527-ad81-7809ac20d270'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:00.425098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] The token cache is being invalidate because project c487dbdcde1f4ef098cd8bab7a1b7e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:57:00.425337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:00.425921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-8095fdf2-04a5-4527-ad81-7809ac20d270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:00.446966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1214/2432] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:56:59 2022] DELETE /identity/v3/projects/c487dbdcde1f4ef098cd8bab7a1b7e27 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:00.863216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:00.864793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:00.879553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf457a6-a064-4626-936a-82faa69d9692 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:00.932080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:00.955888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:00.956552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:00.956824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:00.957095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] PATH_INFO: `/v3/projects/859ace971b1d427ba987371a90db9587` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:00.958258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=859ace971b1d427ba987371a90db9587)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:00.960707 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:01.095012 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '859ace971b1d427ba987371a90db9587', 'request_id': 'req-556c23a6-e38b-4592-81d3-3828bdab02ad'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:01.102571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '859ace971b1d427ba987371a90db9587', 'request_id': 'req-556c23a6-e38b-4592-81d3-3828bdab02ad'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:01.229592 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '859ace971b1d427ba987371a90db9587', 'request_id': 'req-556c23a6-e38b-4592-81d3-3828bdab02ad'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:01.242905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] The token cache is being invalidate because project 859ace971b1d427ba987371a90db9587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:57:01.243175 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:01.243927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-556c23a6-e38b-4592-81d3-3828bdab02ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:01.245530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1219/2433] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:57:00 2022] DELETE /identity/v3/projects/859ace971b1d427ba987371a90db9587 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:01.250622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4706477-2acd-43eb-b45d-709a5874fa7d None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:01.251210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4706477-2acd-43eb-b45d-709a5874fa7d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:01.251210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4706477-2acd-43eb-b45d-709a5874fa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:01.274682 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f4706477-2acd-43eb-b45d-709a5874fa7d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:01.275938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f4706477-2acd-43eb-b45d-709a5874fa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:01.352993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1215/2434] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:01.361447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045244d-c44f-43b6-889a-097030db7e46 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:01.361755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045244d-c44f-43b6-889a-097030db7e46 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:01.362014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045244d-c44f-43b6-889a-097030db7e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:01.385446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a045244d-c44f-43b6-889a-097030db7e46 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:01.386674 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a045244d-c44f-43b6-889a-097030db7e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:01.396339 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1220/2435] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:01.403898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4706477-2acd-43eb-b45d-709a5874fa7d None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:01.423247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4706477-2acd-43eb-b45d-709a5874fa7d None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:01.430263 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce34bdf-7404-4c26-8e6b-395e3b318044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:01.430934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce34bdf-7404-4c26-8e6b-395e3b318044 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:01.431212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce34bdf-7404-4c26-8e6b-395e3b318044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:01.431484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce34bdf-7404-4c26-8e6b-395e3b318044 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:01.432162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce34bdf-7404-4c26-8e6b-395e3b318044 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:57:01.432476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce34bdf-7404-4c26-8e6b-395e3b318044 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:01.434277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce34bdf-7404-4c26-8e6b-395e3b318044 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:01.450130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1216/2436] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:57:01.458945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045244d-c44f-43b6-889a-097030db7e46 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:01.472937 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045244d-c44f-43b6-889a-097030db7e46 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:01.480990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19db8c0-b934-4125-942a-427d0f49be95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:01.481777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19db8c0-b934-4125-942a-427d0f49be95 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:01.482059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19db8c0-b934-4125-942a-427d0f49be95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:01.482329 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19db8c0-b934-4125-942a-427d0f49be95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:01.486601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19db8c0-b934-4125-942a-427d0f49be95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:01.486601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19db8c0-b934-4125-942a-427d0f49be95 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:01.629182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1221/2437] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:57:01 2022] POST /identity/v3/projects => generated 391 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:01.636215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce34bdf-7404-4c26-8e6b-395e3b318044 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:01.648721 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce34bdf-7404-4c26-8e6b-395e3b318044 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:01.727784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e54a2d-b5e3-41b1-b27c-f6b6355f56f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:01.728826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e54a2d-b5e3-41b1-b27c-f6b6355f56f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:01.729318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e54a2d-b5e3-41b1-b27c-f6b6355f56f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:01.729715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e54a2d-b5e3-41b1-b27c-f6b6355f56f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:01.730728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e54a2d-b5e3-41b1-b27c-f6b6355f56f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:01.733054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e54a2d-b5e3-41b1-b27c-f6b6355f56f9 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:01.836484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1217/2438] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:57:01 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:57:01.841446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19db8c0-b934-4125-942a-427d0f49be95 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:01.854671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19db8c0-b934-4125-942a-427d0f49be95 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:01.859575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea11d95-19e3-48a3-a149-35bd9687fe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:01.860187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11d95-19e3-48a3-a149-35bd9687fe79 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:01.860460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11d95-19e3-48a3-a149-35bd9687fe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:01.860724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11d95-19e3-48a3-a149-35bd9687fe79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:01.861463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea11d95-19e3-48a3-a149-35bd9687fe79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:01.863410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea11d95-19e3-48a3-a149-35bd9687fe79 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:01.876112 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1222/2439] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:01.880238 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e54a2d-b5e3-41b1-b27c-f6b6355f56f9 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:01.892250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e54a2d-b5e3-41b1-b27c-f6b6355f56f9 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:01.898141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322555e-0f46-48a6-a79d-bd22711c3fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:01.898806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322555e-0f46-48a6-a79d-bd22711c3fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:01.899085 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322555e-0f46-48a6-a79d-bd22711c3fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:01.899384 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322555e-0f46-48a6-a79d-bd22711c3fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:01.900039 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322555e-0f46-48a6-a79d-bd22711c3fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:01.901674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322555e-0f46-48a6-a79d-bd22711c3fc5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:01.913750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1218/2440] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:01.918644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea11d95-19e3-48a3-a149-35bd9687fe79 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:01.929219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea11d95-19e3-48a3-a149-35bd9687fe79 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:01.933993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8df83-56c7-4c27-b91b-4d725b100d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:01.934690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8df83-56c7-4c27-b91b-4d725b100d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:01.934968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8df83-56c7-4c27-b91b-4d725b100d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:01.935257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8df83-56c7-4c27-b91b-4d725b100d06 admin admin] PATH_INFO: `/v3/projects/6c1b8ef3b1a144d7b873bff7fd8403da/users/3ec4d1768e9a4b5a935197ad78444312/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:01.952016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8df83-56c7-4c27-b91b-4d725b100d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1b8ef3b1a144d7b873bff7fd8403da, user_id=3ec4d1768e9a4b5a935197ad78444312, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:01.954790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8df83-56c7-4c27-b91b-4d725b100d06 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:02.041678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1223/2441] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:01 2022] PUT /identity/v3/projects/6c1b8ef3b1a144d7b873bff7fd8403da/users/3ec4d1768e9a4b5a935197ad78444312/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:02.052296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74893bb4-7bbe-4908-a683-6b92f785247b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:02.052577 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74893bb4-7bbe-4908-a683-6b92f785247b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:02.052855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74893bb4-7bbe-4908-a683-6b92f785247b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:02.084590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-74893bb4-7bbe-4908-a683-6b92f785247b None None] MFA Rules not processed for user `3ec4d1768e9a4b5a935197ad78444312`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:02.085942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-74893bb4-7bbe-4908-a683-6b92f785247b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:02.123691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:02.173535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1219/2442] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:02.177881 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8df83-56c7-4c27-b91b-4d725b100d06 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:02.254515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8df83-56c7-4c27-b91b-4d725b100d06 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:02.262011 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e692a1-f486-49c0-a51c-544cf777e98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:02.263091 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e692a1-f486-49c0-a51c-544cf777e98c admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:02.263780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e692a1-f486-49c0-a51c-544cf777e98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:02.264243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e692a1-f486-49c0-a51c-544cf777e98c admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:02.265357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e692a1-f486-49c0-a51c-544cf777e98c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:02.269115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e692a1-f486-49c0-a51c-544cf777e98c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:02.286936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:02.404859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1224/2443] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:57:02 2022] POST /identity/v3/projects => generated 391 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:57:02.409389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74893bb4-7bbe-4908-a683-6b92f785247b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:02.422322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74893bb4-7bbe-4908-a683-6b92f785247b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:02.490136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557bddc-b68f-4a08-9c36-667c4b47bbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:02.490887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557bddc-b68f-4a08-9c36-667c4b47bbf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:02.491432 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557bddc-b68f-4a08-9c36-667c4b47bbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:02.491709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557bddc-b68f-4a08-9c36-667c4b47bbf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:02.492367 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557bddc-b68f-4a08-9c36-667c4b47bbf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:02.494292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557bddc-b68f-4a08-9c36-667c4b47bbf6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:02.580433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1220/2444] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:57:02 2022] POST /identity/v3/users => generated 302 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:02.584794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e692a1-f486-49c0-a51c-544cf777e98c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:02.595895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e692a1-f486-49c0-a51c-544cf777e98c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:02.602818 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b4f37-6b41-4845-8331-f0079adcbdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:02.604170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4f37-6b41-4845-8331-f0079adcbdd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:02.604676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4f37-6b41-4845-8331-f0079adcbdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:02.605174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4f37-6b41-4845-8331-f0079adcbdd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:02.606236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b4f37-6b41-4845-8331-f0079adcbdd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:02.609399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b4f37-6b41-4845-8331-f0079adcbdd1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:02.623357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1225/2445] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:02.628066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557bddc-b68f-4a08-9c36-667c4b47bbf6 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:02.639016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557bddc-b68f-4a08-9c36-667c4b47bbf6 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:02.644483 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f54f3-3f95-46f7-8dec-3a2a3a0a0e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:02.645247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f54f3-3f95-46f7-8dec-3a2a3a0a0e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:02.645415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f54f3-3f95-46f7-8dec-3a2a3a0a0e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:02.645690 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f54f3-3f95-46f7-8dec-3a2a3a0a0e81 admin admin] PATH_INFO: `/v3/projects/15a512809f95473292be4a440a0cec59/users/69a8f75bc1e84aaa8b09d0a9fb6ec50a/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:02.663346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f54f3-3f95-46f7-8dec-3a2a3a0a0e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a512809f95473292be4a440a0cec59, user_id=69a8f75bc1e84aaa8b09d0a9fb6ec50a, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:02.665974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f54f3-3f95-46f7-8dec-3a2a3a0a0e81 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:02.855053 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1221/2446] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:02 2022] PUT /identity/v3/projects/15a512809f95473292be4a440a0cec59/users/69a8f75bc1e84aaa8b09d0a9fb6ec50a/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:02.859529 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b4f37-6b41-4845-8331-f0079adcbdd1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:02.870953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b4f37-6b41-4845-8331-f0079adcbdd1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:02.951383 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87bf18-ef87-4d9d-81b3-4e677b9ecba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:02.951908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87bf18-ef87-4d9d-81b3-4e677b9ecba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:02.952187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87bf18-ef87-4d9d-81b3-4e677b9ecba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:02.952459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87bf18-ef87-4d9d-81b3-4e677b9ecba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:02.953104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87bf18-ef87-4d9d-81b3-4e677b9ecba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:02.954753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87bf18-ef87-4d9d-81b3-4e677b9ecba8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:02.968951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1226/2447] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:02.973409 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f54f3-3f95-46f7-8dec-3a2a3a0a0e81 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:02.985732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f54f3-3f95-46f7-8dec-3a2a3a0a0e81 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:02.990722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1d80e-dcf1-400a-bea0-fcd5e9e740fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:02.991569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1d80e-dcf1-400a-bea0-fcd5e9e740fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:02.991846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1d80e-dcf1-400a-bea0-fcd5e9e740fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:02.992117 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1d80e-dcf1-400a-bea0-fcd5e9e740fe admin admin] PATH_INFO: `/v3/projects/15a512809f95473292be4a440a0cec59/users/69a8f75bc1e84aaa8b09d0a9fb6ec50a/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:02.995030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1d80e-dcf1-400a-bea0-fcd5e9e740fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a512809f95473292be4a440a0cec59, user_id=69a8f75bc1e84aaa8b09d0a9fb6ec50a, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:02.998157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1d80e-dcf1-400a-bea0-fcd5e9e740fe admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:03.012050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1222/2448] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:02 2022] PUT /identity/v3/projects/15a512809f95473292be4a440a0cec59/users/69a8f75bc1e84aaa8b09d0a9fb6ec50a/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:03.024193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e31d6-a425-4c4b-b242-38b9b582ac83 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:03.025037 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e31d6-a425-4c4b-b242-38b9b582ac83 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:03.025037 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e31d6-a425-4c4b-b242-38b9b582ac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:03.066065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e43e31d6-a425-4c4b-b242-38b9b582ac83 None None] MFA Rules not processed for user `69a8f75bc1e84aaa8b09d0a9fb6ec50a`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:03.067338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e43e31d6-a425-4c4b-b242-38b9b582ac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:03.148435 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1227/2449] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:03 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:03.161171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1d80e-dcf1-400a-bea0-fcd5e9e740fe admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:03.163975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8bf1d80e-dcf1-400a-bea0-fcd5e9e740fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:03.241903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1d80e-dcf1-400a-bea0-fcd5e9e740fe admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:03.245642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fae28-197d-4c51-b7cc-992de6e38126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:03.246268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fae28-197d-4c51-b7cc-992de6e38126 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:03.246602 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fae28-197d-4c51-b7cc-992de6e38126 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:03.246871 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fae28-197d-4c51-b7cc-992de6e38126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:03.258020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fae28-197d-4c51-b7cc-992de6e38126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:03.259530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fae28-197d-4c51-b7cc-992de6e38126 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:03.270044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1223/2450] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:57:03.326635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e31d6-a425-4c4b-b242-38b9b582ac83 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:03.328186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e43e31d6-a425-4c4b-b242-38b9b582ac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:03.342698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e31d6-a425-4c4b-b242-38b9b582ac83 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:03.354464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a6dd0-d5d5-4067-ba63-51fc37f2c326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:03.355103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a6dd0-d5d5-4067-ba63-51fc37f2c326 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:03.359933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a6dd0-d5d5-4067-ba63-51fc37f2c326 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:03.359933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a6dd0-d5d5-4067-ba63-51fc37f2c326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:03.435978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a6dd0-d5d5-4067-ba63-51fc37f2c326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:03.437436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a6dd0-d5d5-4067-ba63-51fc37f2c326 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:03.441625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:03.450614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1228/2451] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:57:03.941012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f832b3-76c6-4c53-808b-8f2e14d243c3 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:03.941591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f832b3-76c6-4c53-808b-8f2e14d243c3 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:03.941591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f832b3-76c6-4c53-808b-8f2e14d243c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:03.961880 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-60f832b3-76c6-4c53-808b-8f2e14d243c3 None None] MFA Rules not processed for user `34ef7ee27c3948ac9df1ff94ce24618e`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:03.965160 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-60f832b3-76c6-4c53-808b-8f2e14d243c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:03.977688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:04.031212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1224/2452] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:04.385979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a6dd0-d5d5-4067-ba63-51fc37f2c326 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:04.387813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-876a6dd0-d5d5-4067-ba63-51fc37f2c326 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:04.451860 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a6dd0-d5d5-4067-ba63-51fc37f2c326 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:04.455540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929d01f-33df-420a-99a8-aad46e338a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:04.456138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929d01f-33df-420a-99a8-aad46e338a75 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:04.456518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929d01f-33df-420a-99a8-aad46e338a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:04.456789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929d01f-33df-420a-99a8-aad46e338a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:04.458532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6929d01f-33df-420a-99a8-aad46e338a75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:04.469205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929d01f-33df-420a-99a8-aad46e338a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:04.470909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929d01f-33df-420a-99a8-aad46e338a75 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:04.481183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1229/2453] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:57:08.987479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f832b3-76c6-4c53-808b-8f2e14d243c3 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:09.066513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f832b3-76c6-4c53-808b-8f2e14d243c3 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:09.074242 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:09.074982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:09.075331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:09.075604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] PATH_INFO: `/v3/users/3ec4d1768e9a4b5a935197ad78444312` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:09.079651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec4d1768e9a4b5a935197ad78444312)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:09.079651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:09.326339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec4d1768e9a4b5a935197ad78444312'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:09.343523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec4d1768e9a4b5a935197ad78444312'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:09.355704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec4d1768e9a4b5a935197ad78444312'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:09.447385 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec4d1768e9a4b5a935197ad78444312'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:09.448611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec4d1768e9a4b5a935197ad78444312'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:09.459413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1225/2454] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:57:08 2022] DELETE /identity/v3/users/3ec4d1768e9a4b5a935197ad78444312 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:09.464158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625d59d-1eb4-4ff1-8f94-725859cdfbcb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:09.464686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625d59d-1eb4-4ff1-8f94-725859cdfbcb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:09.464686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625d59d-1eb4-4ff1-8f94-725859cdfbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:09.489059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f625d59d-1eb4-4ff1-8f94-725859cdfbcb None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:09.490612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f625d59d-1eb4-4ff1-8f94-725859cdfbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:09.563094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1230/2455] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:09.574319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:09.575928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:09.643468 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625d59d-1eb4-4ff1-8f94-725859cdfbcb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:09.644456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:09.645543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f625d59d-1eb4-4ff1-8f94-725859cdfbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:09.666848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db89737-9149-4c00-b68c-fb12e953dca5 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:09.671655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e038a-365f-4b90-bf43-589c1a424564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:09.672505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e038a-365f-4b90-bf43-589c1a424564 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:09.672936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e038a-365f-4b90-bf43-589c1a424564 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:09.673311 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e038a-365f-4b90-bf43-589c1a424564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:09.684567 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e038a-365f-4b90-bf43-589c1a424564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:09.686406 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e038a-365f-4b90-bf43-589c1a424564 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:09.711169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1226/2456] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:57:09.769219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625d59d-1eb4-4ff1-8f94-725859cdfbcb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:09.775634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddaa2e-08a0-4d1c-9c9b-a0c0d566517f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:09.776478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddaa2e-08a0-4d1c-9c9b-a0c0d566517f service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:09.776840 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddaa2e-08a0-4d1c-9c9b-a0c0d566517f service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:09.777150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddaa2e-08a0-4d1c-9c9b-a0c0d566517f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:09.779757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-38ddaa2e-08a0-4d1c-9c9b-a0c0d566517f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:09.795324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddaa2e-08a0-4d1c-9c9b-a0c0d566517f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:09.797018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddaa2e-08a0-4d1c-9c9b-a0c0d566517f service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:09.809376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1231/2457] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:57:10.266762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e038a-365f-4b90-bf43-589c1a424564 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:10.268327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6f3e038a-365f-4b90-bf43-589c1a424564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:10.285019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e038a-365f-4b90-bf43-589c1a424564 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:10.291202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:10.293569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:10.293569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:10.293569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] PATH_INFO: `/v3/projects/6c1b8ef3b1a144d7b873bff7fd8403da` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:10.293569 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1b8ef3b1a144d7b873bff7fd8403da)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:10.295577 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:10.465842 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1b8ef3b1a144d7b873bff7fd8403da', 'request_id': 'req-b394eb16-e682-407d-b5b8-67c278a10bef'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:10.577771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1b8ef3b1a144d7b873bff7fd8403da', 'request_id': 'req-b394eb16-e682-407d-b5b8-67c278a10bef'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:10.587439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1b8ef3b1a144d7b873bff7fd8403da', 'request_id': 'req-b394eb16-e682-407d-b5b8-67c278a10bef'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:10.602373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] The token cache is being invalidate because project 6c1b8ef3b1a144d7b873bff7fd8403da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:57:10.602822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:10.603573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:10.605371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1227/2458] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:57:10 2022] DELETE /identity/v3/projects/6c1b8ef3b1a144d7b873bff7fd8403da => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:10.612037 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddaa2e-08a0-4d1c-9c9b-a0c0d566517f service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:10.625630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddaa2e-08a0-4d1c-9c9b-a0c0d566517f service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:10.677446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:10.694049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52034bad-bb2c-42e3-9747-544a85358a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:10.694796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034bad-bb2c-42e3-9747-544a85358a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:10.695212 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034bad-bb2c-42e3-9747-544a85358a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:10.695502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034bad-bb2c-42e3-9747-544a85358a40 admin admin] PATH_INFO: `/v3/users/69a8f75bc1e84aaa8b09d0a9fb6ec50a` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:10.696690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52034bad-bb2c-42e3-9747-544a85358a40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a8f75bc1e84aaa8b09d0a9fb6ec50a)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:10.698702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52034bad-bb2c-42e3-9747-544a85358a40 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:11.021785 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-52034bad-bb2c-42e3-9747-544a85358a40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a8f75bc1e84aaa8b09d0a9fb6ec50a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:11.035432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-52034bad-bb2c-42e3-9747-544a85358a40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a8f75bc1e84aaa8b09d0a9fb6ec50a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:11.044777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-52034bad-bb2c-42e3-9747-544a85358a40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a8f75bc1e84aaa8b09d0a9fb6ec50a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:11.190101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-52034bad-bb2c-42e3-9747-544a85358a40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a8f75bc1e84aaa8b09d0a9fb6ec50a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:11.193553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-52034bad-bb2c-42e3-9747-544a85358a40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a8f75bc1e84aaa8b09d0a9fb6ec50a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:11.202279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1232/2459] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:57:10 2022] DELETE /identity/v3/users/69a8f75bc1e84aaa8b09d0a9fb6ec50a => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:11.797210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:11.798864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:11.812336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394eb16-e682-407d-b5b8-67c278a10bef admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:11.873649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:11.874398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:11.874709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:11.874959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] PATH_INFO: `/v3/projects/15a512809f95473292be4a440a0cec59` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:11.882534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a512809f95473292be4a440a0cec59)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:11.882534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:12.054318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a512809f95473292be4a440a0cec59', 'request_id': 'req-24996b8f-d48b-45c9-8bdd-b716151a6a53'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:12.156264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a512809f95473292be4a440a0cec59', 'request_id': 'req-24996b8f-d48b-45c9-8bdd-b716151a6a53'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:12.167593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a512809f95473292be4a440a0cec59', 'request_id': 'req-24996b8f-d48b-45c9-8bdd-b716151a6a53'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:12.178459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:12.183824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] The token cache is being invalidate because project 15a512809f95473292be4a440a0cec59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:57:12.184401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:12.185620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-24996b8f-d48b-45c9-8bdd-b716151a6a53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:12.190802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1228/2460] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:57:11 2022] DELETE /identity/v3/projects/15a512809f95473292be4a440a0cec59 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:12.198256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34825c-89df-4402-b3da-01fa0fa36a43 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:12.199739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34825c-89df-4402-b3da-01fa0fa36a43 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:12.205119 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34825c-89df-4402-b3da-01fa0fa36a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:12.237233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ee34825c-89df-4402-b3da-01fa0fa36a43 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:12.238544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ee34825c-89df-4402-b3da-01fa0fa36a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:12.304791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1233/2461] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:12.314300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfaada-0b5c-4121-93aa-e05ebdb1d3a4 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:12.314965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfaada-0b5c-4121-93aa-e05ebdb1d3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:12.315484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfaada-0b5c-4121-93aa-e05ebdb1d3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:12.350813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-28bfaada-0b5c-4121-93aa-e05ebdb1d3a4 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:12.354692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-28bfaada-0b5c-4121-93aa-e05ebdb1d3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:12.368784 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1229/2462] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:57:12.374311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34825c-89df-4402-b3da-01fa0fa36a43 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:12.384545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34825c-89df-4402-b3da-01fa0fa36a43 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:12.389604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e9019-4f3a-4bb4-8e15-7478261fcf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:12.390184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e9019-4f3a-4bb4-8e15-7478261fcf7a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:12.390493 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e9019-4f3a-4bb4-8e15-7478261fcf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:12.390765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e9019-4f3a-4bb4-8e15-7478261fcf7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:12.391445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e9019-4f3a-4bb4-8e15-7478261fcf7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:57:12.391742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e9019-4f3a-4bb4-8e15-7478261fcf7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:12.393702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e9019-4f3a-4bb4-8e15-7478261fcf7a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:12.407417 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1234/2463] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:57:12.411991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfaada-0b5c-4121-93aa-e05ebdb1d3a4 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:12.422552 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfaada-0b5c-4121-93aa-e05ebdb1d3a4 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:12.427634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05978f63-db7c-4397-af96-bc5c1845046d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:12.428233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05978f63-db7c-4397-af96-bc5c1845046d admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:12.428509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05978f63-db7c-4397-af96-bc5c1845046d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:12.429718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05978f63-db7c-4397-af96-bc5c1845046d admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:12.431036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05978f63-db7c-4397-af96-bc5c1845046d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:12.432960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05978f63-db7c-4397-af96-bc5c1845046d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:12.529426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1230/2464] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:57:12 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:12.534146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e9019-4f3a-4bb4-8e15-7478261fcf7a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:12.546559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e9019-4f3a-4bb4-8e15-7478261fcf7a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:12.596045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:12.606408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2b6b9-240e-47e5-9873-e245d6ad2be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:12.607046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2b6b9-240e-47e5-9873-e245d6ad2be5 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:12.607322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2b6b9-240e-47e5-9873-e245d6ad2be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:12.607590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2b6b9-240e-47e5-9873-e245d6ad2be5 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:12.608247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2b6b9-240e-47e5-9873-e245d6ad2be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:12.610166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2b6b9-240e-47e5-9873-e245d6ad2be5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:12.790959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1235/2465] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:57:12 2022] POST /identity/v3/users => generated 287 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:12.795553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05978f63-db7c-4397-af96-bc5c1845046d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:12.807583 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05978f63-db7c-4397-af96-bc5c1845046d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:12.813157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd98035-4738-43a3-9e70-8876241ec0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:12.813783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98035-4738-43a3-9e70-8876241ec0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:12.814063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98035-4738-43a3-9e70-8876241ec0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:12.814360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98035-4738-43a3-9e70-8876241ec0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:12.815532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd98035-4738-43a3-9e70-8876241ec0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:12.817028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd98035-4738-43a3-9e70-8876241ec0e7 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:12.828677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1231/2466] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:12.832642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2b6b9-240e-47e5-9873-e245d6ad2be5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:12.847489 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2b6b9-240e-47e5-9873-e245d6ad2be5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:12.854491 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c6add-254a-47fe-8b8d-9c447bd95fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:12.855400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c6add-254a-47fe-8b8d-9c447bd95fda admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:12.855812 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c6add-254a-47fe-8b8d-9c447bd95fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:12.856180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c6add-254a-47fe-8b8d-9c447bd95fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:12.856970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c6add-254a-47fe-8b8d-9c447bd95fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:12.858861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c6add-254a-47fe-8b8d-9c447bd95fda admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:12.872504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1236/2467] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:57:12.876728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd98035-4738-43a3-9e70-8876241ec0e7 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:12.888230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd98035-4738-43a3-9e70-8876241ec0e7 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:12.892989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198c08b-276e-4347-b075-4e18e69fb142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:12.893641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198c08b-276e-4347-b075-4e18e69fb142 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:12.894407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198c08b-276e-4347-b075-4e18e69fb142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:12.894795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198c08b-276e-4347-b075-4e18e69fb142 admin admin] PATH_INFO: `/v3/projects/7705da6859424c31bc3090483195b2e0/users/02d00899a62b42a695f886d129c942d8/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:12.912419 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198c08b-276e-4347-b075-4e18e69fb142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7705da6859424c31bc3090483195b2e0, user_id=02d00899a62b42a695f886d129c942d8, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:12.916362 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198c08b-276e-4347-b075-4e18e69fb142 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:13.024407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1232/2468] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:12 2022] PUT /identity/v3/projects/7705da6859424c31bc3090483195b2e0/users/02d00899a62b42a695f886d129c942d8/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:13.030775 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0ebb1-1955-4b15-b228-38e1beb81df4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:13.031170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0ebb1-1955-4b15-b228-38e1beb81df4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:13.031510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0ebb1-1955-4b15-b228-38e1beb81df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:13.052256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-45b0ebb1-1955-4b15-b228-38e1beb81df4 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:13.053499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-45b0ebb1-1955-4b15-b228-38e1beb81df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:13.118895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1237/2469] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:57:13.129307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198c08b-276e-4347-b075-4e18e69fb142 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:13.131172 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4198c08b-276e-4347-b075-4e18e69fb142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:13.195532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198c08b-276e-4347-b075-4e18e69fb142 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:13.199318 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc26be4-852b-45e4-863e-c7b52f795433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:13.199935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26be4-852b-45e4-863e-c7b52f795433 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:13.200470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26be4-852b-45e4-863e-c7b52f795433 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:13.200470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26be4-852b-45e4-863e-c7b52f795433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:13.210467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc26be4-852b-45e4-863e-c7b52f795433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:13.211925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc26be4-852b-45e4-863e-c7b52f795433 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:13.224428 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1233/2470] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:57:14.742490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e1829-4962-4bf5-94e2-21176e7b0d37 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:14.749093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e1829-4962-4bf5-94e2-21176e7b0d37 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:14.749093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e1829-4962-4bf5-94e2-21176e7b0d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:14.803617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ec2e1829-4962-4bf5-94e2-21176e7b0d37 None None] MFA Rules not processed for user `02d00899a62b42a695f886d129c942d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:14.806038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ec2e1829-4962-4bf5-94e2-21176e7b0d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:14.937291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1238/2471] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:14.942093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc26be4-852b-45e4-863e-c7b52f795433 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:14.959661 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc26be4-852b-45e4-863e-c7b52f795433 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:14.966622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb281af-bf5c-4a9f-8d09-b58345140a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:14.967611 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb281af-bf5c-4a9f-8d09-b58345140a12 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:14.968023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb281af-bf5c-4a9f-8d09-b58345140a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:14.968408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb281af-bf5c-4a9f-8d09-b58345140a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:14.969473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb281af-bf5c-4a9f-8d09-b58345140a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:14.972032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb281af-bf5c-4a9f-8d09-b58345140a12 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:15.052646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e1829-4962-4bf5-94e2-21176e7b0d37 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:15.055447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ec2e1829-4962-4bf5-94e2-21176e7b0d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:15.086032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1234/2472] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:57:14 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:15.091023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb281af-bf5c-4a9f-8d09-b58345140a12 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:15.136651 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:15.155827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e1829-4962-4bf5-94e2-21176e7b0d37 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:15.207320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb281af-bf5c-4a9f-8d09-b58345140a12 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:15.212868 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af9474-e2e9-4f29-a0c4-9fdf786b4b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:15.213493 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af9474-e2e9-4f29-a0c4-9fdf786b4b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:15.213782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af9474-e2e9-4f29-a0c4-9fdf786b4b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:15.214060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af9474-e2e9-4f29-a0c4-9fdf786b4b53 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:15.214756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af9474-e2e9-4f29-a0c4-9fdf786b4b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:15.216797 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af9474-e2e9-4f29-a0c4-9fdf786b4b53 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:15.254260 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f5d61-1e38-4f1a-8a7c-07ee86c0361d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:15.254940 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f5d61-1e38-4f1a-8a7c-07ee86c0361d service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:15.255218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f5d61-1e38-4f1a-8a7c-07ee86c0361d service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:15.255494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f5d61-1e38-4f1a-8a7c-07ee86c0361d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:15.258301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a46f5d61-1e38-4f1a-8a7c-07ee86c0361d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:15.272024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f5d61-1e38-4f1a-8a7c-07ee86c0361d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:15.273841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f5d61-1e38-4f1a-8a7c-07ee86c0361d service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:15.315235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1239/2473] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:57:15.321612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1235/2474] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:57:15 2022] POST /identity/v3/users => generated 292 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:57:15.328462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f5d61-1e38-4f1a-8a7c-07ee86c0361d service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:15.358160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f5d61-1e38-4f1a-8a7c-07ee86c0361d service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:15.370649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421353f4-2b86-4f58-8acf-2e6fa5de9c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:15.372116 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421353f4-2b86-4f58-8acf-2e6fa5de9c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:15.372622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421353f4-2b86-4f58-8acf-2e6fa5de9c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:15.373252 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421353f4-2b86-4f58-8acf-2e6fa5de9c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:15.374624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421353f4-2b86-4f58-8acf-2e6fa5de9c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:15.378096 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421353f4-2b86-4f58-8acf-2e6fa5de9c7d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:15.461662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1240/2475] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:15.465192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af9474-e2e9-4f29-a0c4-9fdf786b4b53 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:15.477702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af9474-e2e9-4f29-a0c4-9fdf786b4b53 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:15.482739 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247c315-13e2-4235-9cce-cea7b419d9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:15.483506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247c315-13e2-4235-9cce-cea7b419d9e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:15.483787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247c315-13e2-4235-9cce-cea7b419d9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:15.484064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247c315-13e2-4235-9cce-cea7b419d9e5 admin admin] PATH_INFO: `/v3/projects/d47787ed2a1f4a36b6be8a9a3ba4df77/users/e2b7f0d5c1b34c21b176f3b02372a5dc/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:15.486828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:15.502310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247c315-13e2-4235-9cce-cea7b419d9e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47787ed2a1f4a36b6be8a9a3ba4df77, user_id=e2b7f0d5c1b34c21b176f3b02372a5dc, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:15.505182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247c315-13e2-4235-9cce-cea7b419d9e5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:15.701006 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1236/2476] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:15 2022] PUT /identity/v3/projects/d47787ed2a1f4a36b6be8a9a3ba4df77/users/e2b7f0d5c1b34c21b176f3b02372a5dc/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:15.706403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421353f4-2b86-4f58-8acf-2e6fa5de9c7d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:15.717399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421353f4-2b86-4f58-8acf-2e6fa5de9c7d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:15.791548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838bb23-1a27-476c-81a0-746633b32a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:15.792619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838bb23-1a27-476c-81a0-746633b32a76 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:15.792935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838bb23-1a27-476c-81a0-746633b32a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:15.793547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838bb23-1a27-476c-81a0-746633b32a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:15.794318 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838bb23-1a27-476c-81a0-746633b32a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:15.798985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838bb23-1a27-476c-81a0-746633b32a76 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:15.808549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1241/2477] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:15.815468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247c315-13e2-4235-9cce-cea7b419d9e5 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:15.834691 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247c315-13e2-4235-9cce-cea7b419d9e5 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:15.844331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862727a-dd26-4670-8786-a44e0899171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:15.845084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862727a-dd26-4670-8786-a44e0899171b admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:15.845418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862727a-dd26-4670-8786-a44e0899171b admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:15.845847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862727a-dd26-4670-8786-a44e0899171b admin admin] PATH_INFO: `/v3/projects/d47787ed2a1f4a36b6be8a9a3ba4df77/users/e2b7f0d5c1b34c21b176f3b02372a5dc/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:15.848655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862727a-dd26-4670-8786-a44e0899171b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47787ed2a1f4a36b6be8a9a3ba4df77, user_id=e2b7f0d5c1b34c21b176f3b02372a5dc, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:15.853597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862727a-dd26-4670-8786-a44e0899171b admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:15.874952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1237/2478] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:15 2022] PUT /identity/v3/projects/d47787ed2a1f4a36b6be8a9a3ba4df77/users/e2b7f0d5c1b34c21b176f3b02372a5dc/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:17.684705 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a48c9-3d94-4e34-a95c-b77171c94ba3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:17.685202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a48c9-3d94-4e34-a95c-b77171c94ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:17.685202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a48c9-3d94-4e34-a95c-b77171c94ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:17.715394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0d0a48c9-3d94-4e34-a95c-b77171c94ba3 None None] MFA Rules not processed for user `e2b7f0d5c1b34c21b176f3b02372a5dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:17.721173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0d0a48c9-3d94-4e34-a95c-b77171c94ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:17.792315 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:17.805722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1242/2479] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:17.817136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862727a-dd26-4670-8786-a44e0899171b admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:17.819093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5862727a-dd26-4670-8786-a44e0899171b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:17.884617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862727a-dd26-4670-8786-a44e0899171b admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:17.889075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f231d-c656-49ca-a6a3-0ee8969e52fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:17.889680 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f231d-c656-49ca-a6a3-0ee8969e52fc service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:17.890013 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f231d-c656-49ca-a6a3-0ee8969e52fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:17.890230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f231d-c656-49ca-a6a3-0ee8969e52fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:17.953978 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f231d-c656-49ca-a6a3-0ee8969e52fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:17.955383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f231d-c656-49ca-a6a3-0ee8969e52fc service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:17.965729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1238/2480] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:57:19.792945 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a48c9-3d94-4e34-a95c-b77171c94ba3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:19.794543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0d0a48c9-3d94-4e34-a95c-b77171c94ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:19.808521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a48c9-3d94-4e34-a95c-b77171c94ba3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:19.812687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f39165-3bce-4218-b931-c0114d7d7af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:19.813400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f39165-3bce-4218-b931-c0114d7d7af4 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:19.813760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f39165-3bce-4218-b931-c0114d7d7af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:19.814088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f39165-3bce-4218-b931-c0114d7d7af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:19.826182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f39165-3bce-4218-b931-c0114d7d7af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:19.827692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f39165-3bce-4218-b931-c0114d7d7af4 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:19.839464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1243/2481] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:57:27.961605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=60,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:29.833478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=9 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:36.045886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f231d-c656-49ca-a6a3-0ee8969e52fc service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:36.110062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f231d-c656-49ca-a6a3-0ee8969e52fc service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:36.121159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de822edc-df40-4362-80ee-77f64c57166a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:36.121963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de822edc-df40-4362-80ee-77f64c57166a service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:36.122471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de822edc-df40-4362-80ee-77f64c57166a service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:36.122966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de822edc-df40-4362-80ee-77f64c57166a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:36.126912 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-de822edc-df40-4362-80ee-77f64c57166a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:36.217061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de822edc-df40-4362-80ee-77f64c57166a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:36.218543 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de822edc-df40-4362-80ee-77f64c57166a service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:36.229900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1239/2482] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:57:37.075789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f39165-3bce-4218-b931-c0114d7d7af4 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:37.091696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f39165-3bce-4218-b931-c0114d7d7af4 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:37.097238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:37.097939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:37.098217 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:37.098530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] PATH_INFO: `/v3/users/02d00899a62b42a695f886d129c942d8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:37.100782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02d00899a62b42a695f886d129c942d8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:37.103975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:37.297774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02d00899a62b42a695f886d129c942d8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:37.317588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02d00899a62b42a695f886d129c942d8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:37.326015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02d00899a62b42a695f886d129c942d8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:37.373637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02d00899a62b42a695f886d129c942d8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:37.374384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02d00899a62b42a695f886d129c942d8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:37.386953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1244/2483] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:57:37 2022] DELETE /identity/v3/users/02d00899a62b42a695f886d129c942d8 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:37.716534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de822edc-df40-4362-80ee-77f64c57166a service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:37.727615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de822edc-df40-4362-80ee-77f64c57166a service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:37.799308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:37.799949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:37.800233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:37.800511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] PATH_INFO: `/v3/projects/7705da6859424c31bc3090483195b2e0` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:37.801683 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7705da6859424c31bc3090483195b2e0)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:37.803628 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:37.892674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7705da6859424c31bc3090483195b2e0', 'request_id': 'req-abf02cce-709a-4f72-abb5-dd3a84cfca79'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:37.896145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:38.012468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7705da6859424c31bc3090483195b2e0', 'request_id': 'req-abf02cce-709a-4f72-abb5-dd3a84cfca79'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:38.019698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7705da6859424c31bc3090483195b2e0', 'request_id': 'req-abf02cce-709a-4f72-abb5-dd3a84cfca79'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:38.036264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] The token cache is being invalidate because project 7705da6859424c31bc3090483195b2e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:57:38.036677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:38.037559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:38.039741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1240/2484] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:57:37 2022] DELETE /identity/v3/projects/7705da6859424c31bc3090483195b2e0 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:38.044792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:38.046606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:38.064919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405a601-b485-45d4-9117-ecab821bd489 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:38.141875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54bc91-8f59-406b-939a-7e3fc38f1f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:38.142572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54bc91-8f59-406b-939a-7e3fc38f1f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:38.142930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54bc91-8f59-406b-939a-7e3fc38f1f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:38.143151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54bc91-8f59-406b-939a-7e3fc38f1f79 admin admin] PATH_INFO: `/v3/users/e2b7f0d5c1b34c21b176f3b02372a5dc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:38.144515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54bc91-8f59-406b-939a-7e3fc38f1f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2b7f0d5c1b34c21b176f3b02372a5dc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:38.147138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54bc91-8f59-406b-939a-7e3fc38f1f79 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:38.377482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cd54bc91-8f59-406b-939a-7e3fc38f1f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2b7f0d5c1b34c21b176f3b02372a5dc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:38.392548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cd54bc91-8f59-406b-939a-7e3fc38f1f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2b7f0d5c1b34c21b176f3b02372a5dc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:38.403559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cd54bc91-8f59-406b-939a-7e3fc38f1f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2b7f0d5c1b34c21b176f3b02372a5dc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:38.403559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:38.480885 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cd54bc91-8f59-406b-939a-7e3fc38f1f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2b7f0d5c1b34c21b176f3b02372a5dc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:38.481619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cd54bc91-8f59-406b-939a-7e3fc38f1f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2b7f0d5c1b34c21b176f3b02372a5dc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:38.488478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1245/2485] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:57:38 2022] DELETE /identity/v3/users/e2b7f0d5c1b34c21b176f3b02372a5dc => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:38.973990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:38.975560 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:38.993395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf02cce-709a-4f72-abb5-dd3a84cfca79 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:39.070035 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:39.070669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:39.070906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:39.071208 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] PATH_INFO: `/v3/projects/d47787ed2a1f4a36b6be8a9a3ba4df77` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:39.073192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d47787ed2a1f4a36b6be8a9a3ba4df77)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:39.075164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:39.193947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd47787ed2a1f4a36b6be8a9a3ba4df77', 'request_id': 'req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:39.281036 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd47787ed2a1f4a36b6be8a9a3ba4df77', 'request_id': 'req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:39.288647 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd47787ed2a1f4a36b6be8a9a3ba4df77', 'request_id': 'req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:39.301127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] The token cache is being invalidate because project d47787ed2a1f4a36b6be8a9a3ba4df77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:57:39.301419 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:39.302127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-035024c8-42c2-47d9-b1c1-7fcc4c2bb679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:39.303730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1241/2486] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:57:38 2022] DELETE /identity/v3/projects/d47787ed2a1f4a36b6be8a9a3ba4df77 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:57:39.311583 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f17e00-bf93-43b5-a9f4-21ad7c639046 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:39.312128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f17e00-bf93-43b5-a9f4-21ad7c639046 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:39.312618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f17e00-bf93-43b5-a9f4-21ad7c639046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:39.347105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c9f17e00-bf93-43b5-a9f4-21ad7c639046 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:39.348526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c9f17e00-bf93-43b5-a9f4-21ad7c639046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:39.425824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1246/2487] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:39.434532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645e0f6-af95-46bf-8a2f-b75d6ca3da2a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:39.434815 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645e0f6-af95-46bf-8a2f-b75d6ca3da2a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:39.435087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645e0f6-af95-46bf-8a2f-b75d6ca3da2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:39.458587 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-8645e0f6-af95-46bf-8a2f-b75d6ca3da2a None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:39.459824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8645e0f6-af95-46bf-8a2f-b75d6ca3da2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:39.469202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1242/2488] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:39.474003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f17e00-bf93-43b5-a9f4-21ad7c639046 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:39.484772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f17e00-bf93-43b5-a9f4-21ad7c639046 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:39.489650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac46a25-1d6f-4acb-aa5f-0b70679d0448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:39.490371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac46a25-1d6f-4acb-aa5f-0b70679d0448 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:39.490695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac46a25-1d6f-4acb-aa5f-0b70679d0448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:39.490970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac46a25-1d6f-4acb-aa5f-0b70679d0448 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:39.491622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac46a25-1d6f-4acb-aa5f-0b70679d0448 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:57:39.491927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac46a25-1d6f-4acb-aa5f-0b70679d0448 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:39.493578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac46a25-1d6f-4acb-aa5f-0b70679d0448 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:39.508151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1247/2489] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:57:39.512367 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645e0f6-af95-46bf-8a2f-b75d6ca3da2a None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:39.524668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645e0f6-af95-46bf-8a2f-b75d6ca3da2a None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:39.530129 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a13e1-b9bc-4188-8012-42e018f917e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:39.530848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a13e1-b9bc-4188-8012-42e018f917e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:39.531131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a13e1-b9bc-4188-8012-42e018f917e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:39.531405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a13e1-b9bc-4188-8012-42e018f917e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:39.532071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a13e1-b9bc-4188-8012-42e018f917e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:39.534014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a13e1-b9bc-4188-8012-42e018f917e9 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:39.627260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1243/2490] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:57:39 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:39.631641 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac46a25-1d6f-4acb-aa5f-0b70679d0448 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:39.642628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac46a25-1d6f-4acb-aa5f-0b70679d0448 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:39.704692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25c74d-1ae1-4d1d-8e63-40347764cdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:39.705361 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25c74d-1ae1-4d1d-8e63-40347764cdbd admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:39.705669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25c74d-1ae1-4d1d-8e63-40347764cdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:39.705946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25c74d-1ae1-4d1d-8e63-40347764cdbd admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:39.706610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25c74d-1ae1-4d1d-8e63-40347764cdbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:39.708535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25c74d-1ae1-4d1d-8e63-40347764cdbd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:39.842894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1248/2491] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:57:39 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:39.843544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a13e1-b9bc-4188-8012-42e018f917e9 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:39.856969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a13e1-b9bc-4188-8012-42e018f917e9 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:39.866841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb722ad-52e5-4562-ba1e-3aea5fee8f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:39.867575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb722ad-52e5-4562-ba1e-3aea5fee8f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:39.868061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb722ad-52e5-4562-ba1e-3aea5fee8f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:39.868496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb722ad-52e5-4562-ba1e-3aea5fee8f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:39.869387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb722ad-52e5-4562-ba1e-3aea5fee8f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:39.871341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb722ad-52e5-4562-ba1e-3aea5fee8f1f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:39.882858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1244/2492] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:39.891310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25c74d-1ae1-4d1d-8e63-40347764cdbd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:39.905744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25c74d-1ae1-4d1d-8e63-40347764cdbd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:39.911404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d2761-0a72-40a8-adec-f49aef788099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:39.912524 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d2761-0a72-40a8-adec-f49aef788099 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:39.912953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d2761-0a72-40a8-adec-f49aef788099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:39.913343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d2761-0a72-40a8-adec-f49aef788099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:39.914263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d2761-0a72-40a8-adec-f49aef788099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:39.916424 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d2761-0a72-40a8-adec-f49aef788099 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:39.932494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1249/2493] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:39.938162 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb722ad-52e5-4562-ba1e-3aea5fee8f1f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:39.951873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb722ad-52e5-4562-ba1e-3aea5fee8f1f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:39.956775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa8bb7-df2b-44c4-ac57-efd47b065397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:39.958252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa8bb7-df2b-44c4-ac57-efd47b065397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:39.958591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa8bb7-df2b-44c4-ac57-efd47b065397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:39.958878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa8bb7-df2b-44c4-ac57-efd47b065397 admin admin] PATH_INFO: `/v3/projects/95f4b3c88d954485bbdb1450b8da4459/users/bb8a66cea871471b842d1379fd9f5916/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:39.978947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa8bb7-df2b-44c4-ac57-efd47b065397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f4b3c88d954485bbdb1450b8da4459, user_id=bb8a66cea871471b842d1379fd9f5916, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:39.981636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa8bb7-df2b-44c4-ac57-efd47b065397 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:40.069116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1245/2494] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:39 2022] PUT /identity/v3/projects/95f4b3c88d954485bbdb1450b8da4459/users/bb8a66cea871471b842d1379fd9f5916/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:40.070773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85bbee-f89e-4cc7-997b-51f4f5cf12a5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:40.071064 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85bbee-f89e-4cc7-997b-51f4f5cf12a5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:40.071342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85bbee-f89e-4cc7-997b-51f4f5cf12a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:40.118967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9f85bbee-f89e-4cc7-997b-51f4f5cf12a5 None None] MFA Rules not processed for user `bb8a66cea871471b842d1379fd9f5916`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:40.120645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9f85bbee-f89e-4cc7-997b-51f4f5cf12a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:40.184901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:40.207135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1250/2495] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:40.219584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa8bb7-df2b-44c4-ac57-efd47b065397 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:40.221388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-03fa8bb7-df2b-44c4-ac57-efd47b065397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:40.233491 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:40.300207 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa8bb7-df2b-44c4-ac57-efd47b065397 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:40.304426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c393b-deac-43a0-831a-cc1b1952830b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:40.312553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c393b-deac-43a0-831a-cc1b1952830b service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:40.312553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c393b-deac-43a0-831a-cc1b1952830b service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:40.312553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c393b-deac-43a0-831a-cc1b1952830b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:40.317673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c393b-deac-43a0-831a-cc1b1952830b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:40.323953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c393b-deac-43a0-831a-cc1b1952830b service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:40.331200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1246/2496] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:57:40.362392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85bbee-f89e-4cc7-997b-51f4f5cf12a5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:40.451293 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85bbee-f89e-4cc7-997b-51f4f5cf12a5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:40.454277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c393b-deac-43a0-831a-cc1b1952830b service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:40.457181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8fbd7-8e84-49b2-a7f2-28b13c1adbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:40.457898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8fbd7-8e84-49b2-a7f2-28b13c1adbab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:40.458188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8fbd7-8e84-49b2-a7f2-28b13c1adbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:40.458495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8fbd7-8e84-49b2-a7f2-28b13c1adbab admin admin] PATH_INFO: `/v3/users/bb8a66cea871471b842d1379fd9f5916` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:40.459469 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bf4c393b-deac-43a0-831a-cc1b1952830b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:40.459726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8fbd7-8e84-49b2-a7f2-28b13c1adbab admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb8a66cea871471b842d1379fd9f5916)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:40.461699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8fbd7-8e84-49b2-a7f2-28b13c1adbab admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:40.549516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c393b-deac-43a0-831a-cc1b1952830b service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:40.554788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51179271-002e-47c0-bd08-fd08d2bbaf8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:40.555414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51179271-002e-47c0-bd08-fd08d2bbaf8c service placement] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:40.555696 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51179271-002e-47c0-bd08-fd08d2bbaf8c service placement] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:40.555967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51179271-002e-47c0-bd08-fd08d2bbaf8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:40.558329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-51179271-002e-47c0-bd08-fd08d2bbaf8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:40.574998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51179271-002e-47c0-bd08-fd08d2bbaf8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:40.576591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51179271-002e-47c0-bd08-fd08d2bbaf8c service placement] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:40.590265 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1247/2497] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:57:40.614667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51179271-002e-47c0-bd08-fd08d2bbaf8c service placement] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:40.617022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-51179271-002e-47c0-bd08-fd08d2bbaf8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:40.640368 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51179271-002e-47c0-bd08-fd08d2bbaf8c service placement] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:40.645753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:40.646517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:40.646883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:40.647273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] PATH_INFO: `/v3/users/db604b7af623452fac9b3712b6978168` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:40.649347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db604b7af623452fac9b3712b6978168)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:40.651900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:40.709684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-36b8fbd7-8e84-49b2-a7f2-28b13c1adbab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb8a66cea871471b842d1379fd9f5916'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:40.724182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-36b8fbd7-8e84-49b2-a7f2-28b13c1adbab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb8a66cea871471b842d1379fd9f5916'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:40.734362 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-36b8fbd7-8e84-49b2-a7f2-28b13c1adbab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb8a66cea871471b842d1379fd9f5916'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:40.800861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-36b8fbd7-8e84-49b2-a7f2-28b13c1adbab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb8a66cea871471b842d1379fd9f5916'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:40.801409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-36b8fbd7-8e84-49b2-a7f2-28b13c1adbab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb8a66cea871471b842d1379fd9f5916'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:40.808946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1251/2498] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:57:40 2022] DELETE /identity/v3/users/bb8a66cea871471b842d1379fd9f5916 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:40.813502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92594e4-f1d0-4c81-8b98-c297881973fb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:40.813787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92594e4-f1d0-4c81-8b98-c297881973fb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:40.814057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92594e4-f1d0-4c81-8b98-c297881973fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:40.839881 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b92594e4-f1d0-4c81-8b98-c297881973fb None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:40.841121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b92594e4-f1d0-4c81-8b98-c297881973fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:40.854706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db604b7af623452fac9b3712b6978168'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:40.869621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db604b7af623452fac9b3712b6978168'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:40.877677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db604b7af623452fac9b3712b6978168'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:40.911168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1252/2499] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:57:40.923464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92594e4-f1d0-4c81-8b98-c297881973fb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:40.925181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b92594e4-f1d0-4c81-8b98-c297881973fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:40.937393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:40.983232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db604b7af623452fac9b3712b6978168'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:40.985308 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db604b7af623452fac9b3712b6978168'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:40.993861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92594e4-f1d0-4c81-8b98-c297881973fb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:40.996389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1248/2500] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:57:40 2022] DELETE /identity/v3/users/db604b7af623452fac9b3712b6978168 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:57:41.072262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896b4bf-71c3-4be5-b92b-f9b55289e876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:41.072926 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896b4bf-71c3-4be5-b92b-f9b55289e876 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:41.073207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896b4bf-71c3-4be5-b92b-f9b55289e876 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:41.073454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896b4bf-71c3-4be5-b92b-f9b55289e876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:41.075430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1896b4bf-71c3-4be5-b92b-f9b55289e876 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:41.161233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896b4bf-71c3-4be5-b92b-f9b55289e876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:41.162795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896b4bf-71c3-4be5-b92b-f9b55289e876 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:41.174602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1253/2501] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:57:41.311822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:41.314388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:41.336361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a704b-6b94-4e65-bdd6-f7fcc3f30402 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:41.342415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:41.343275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:41.343647 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:41.344576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] PATH_INFO: `/v3/projects/2d358f3073d446f49dc51777a7afc30e` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:41.346702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d358f3073d446f49dc51777a7afc30e)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:41.348709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:41.356815 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:41.509329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d358f3073d446f49dc51777a7afc30e', 'request_id': 'req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:41.577914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d358f3073d446f49dc51777a7afc30e', 'request_id': 'req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:41.585250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d358f3073d446f49dc51777a7afc30e', 'request_id': 'req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:41.601941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] The token cache is being invalidate because project 2d358f3073d446f49dc51777a7afc30e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:57:41.602274 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:41.603089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-90b63fe3-a4d5-458e-ad14-b7e0ceea75ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:41.605771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1249/2502] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:57:41 2022] DELETE /identity/v3/projects/2d358f3073d446f49dc51777a7afc30e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:41.613734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f884-c3fd-4267-a4db-cb9ac18023c5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:41.614279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f884-c3fd-4267-a4db-cb9ac18023c5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:41.614279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f884-c3fd-4267-a4db-cb9ac18023c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:41.635466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e828f884-c3fd-4267-a4db-cb9ac18023c5 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:41.636706 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e828f884-c3fd-4267-a4db-cb9ac18023c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:41.698771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1254/2503] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:41.707012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c1f0-6170-4d69-807e-3fe0004ddcf8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:41.707357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c1f0-6170-4d69-807e-3fe0004ddcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:41.707641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c1f0-6170-4d69-807e-3fe0004ddcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:41.733941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-b585c1f0-6170-4d69-807e-3fe0004ddcf8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:41.733941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b585c1f0-6170-4d69-807e-3fe0004ddcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:41.735076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828f884-c3fd-4267-a4db-cb9ac18023c5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:41.736793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e828f884-c3fd-4267-a4db-cb9ac18023c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:41.746537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1250/2504] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:57:41.756675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828f884-c3fd-4267-a4db-cb9ac18023c5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:41.758640 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:41.759487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:41.760311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:41.760589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] PATH_INFO: `/v3/projects/95f4b3c88d954485bbdb1450b8da4459` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:41.761970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f4b3c88d954485bbdb1450b8da4459)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:41.763280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585c1f0-6170-4d69-807e-3fe0004ddcf8 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:41.763974 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:41.773818 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585c1f0-6170-4d69-807e-3fe0004ddcf8 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:41.779462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b431ffb-bb95-4308-ad68-a856e14ef28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:41.779575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b431ffb-bb95-4308-ad68-a856e14ef28c admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:41.780031 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b431ffb-bb95-4308-ad68-a856e14ef28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:41.780669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b431ffb-bb95-4308-ad68-a856e14ef28c admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:41.780883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b431ffb-bb95-4308-ad68-a856e14ef28c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:57:41.781192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b431ffb-bb95-4308-ad68-a856e14ef28c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:41.782900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b431ffb-bb95-4308-ad68-a856e14ef28c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:41.799080 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1251/2505] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:57:41.805522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b431ffb-bb95-4308-ad68-a856e14ef28c admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:41.816540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b431ffb-bb95-4308-ad68-a856e14ef28c admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:41.821761 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfb7aa-06ea-4ad9-ac22-d5183a1400e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:41.823290 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfb7aa-06ea-4ad9-ac22-d5183a1400e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:41.823591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfb7aa-06ea-4ad9-ac22-d5183a1400e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:41.823864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfb7aa-06ea-4ad9-ac22-d5183a1400e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:41.824515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfb7aa-06ea-4ad9-ac22-d5183a1400e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:41.826815 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfb7aa-06ea-4ad9-ac22-d5183a1400e8 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:41.906954 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f4b3c88d954485bbdb1450b8da4459', 'request_id': 'req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:41.913077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f4b3c88d954485bbdb1450b8da4459', 'request_id': 'req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:41.945821 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1252/2506] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:57:41 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:41.947961 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfb7aa-06ea-4ad9-ac22-d5183a1400e8 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.017289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfb7aa-06ea-4ad9-ac22-d5183a1400e8 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:42.022137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3835e-4d1d-4bc5-a68f-543584261239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:42.022839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3835e-4d1d-4bc5-a68f-543584261239 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.023118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3835e-4d1d-4bc5-a68f-543584261239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.023392 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3835e-4d1d-4bc5-a68f-543584261239 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.027044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3835e-4d1d-4bc5-a68f-543584261239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:42.027202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f4b3c88d954485bbdb1450b8da4459', 'request_id': 'req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:42.027336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3835e-4d1d-4bc5-a68f-543584261239 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:42.041567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] The token cache is being invalidate because project 95f4b3c88d954485bbdb1450b8da4459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:57:42.041845 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:42.042587 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dae0a479-e04c-4f92-89aa-1500f2f0f1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:42.044110 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1255/2507] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:57:41 2022] DELETE /identity/v3/projects/95f4b3c88d954485bbdb1450b8da4459 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:57:42.048841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232af12-4d58-4ca9-99ca-442eb0e46226 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.049128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232af12-4d58-4ca9-99ca-442eb0e46226 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.049407 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232af12-4d58-4ca9-99ca-442eb0e46226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.070679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a232af12-4d58-4ca9-99ca-442eb0e46226 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:42.072527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a232af12-4d58-4ca9-99ca-442eb0e46226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:42.080703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1256/2508] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:57:42.088529 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf6aff-ed5e-4e23-a9de-f07be6351b45 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.088807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf6aff-ed5e-4e23-a9de-f07be6351b45 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.089082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf6aff-ed5e-4e23-a9de-f07be6351b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.111158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8abf6aff-ed5e-4e23-a9de-f07be6351b45 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:42.112382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8abf6aff-ed5e-4e23-a9de-f07be6351b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:42.122249 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1257/2509] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:57:42.126606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf6aff-ed5e-4e23-a9de-f07be6351b45 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.139219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf6aff-ed5e-4e23-a9de-f07be6351b45 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:42.144235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8a4cb-5c59-469e-bbec-c40f88726584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:42.144855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8a4cb-5c59-469e-bbec-c40f88726584 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.145145 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8a4cb-5c59-469e-bbec-c40f88726584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.145417 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8a4cb-5c59-469e-bbec-c40f88726584 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.146072 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8a4cb-5c59-469e-bbec-c40f88726584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:57:42.146379 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8a4cb-5c59-469e-bbec-c40f88726584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:42.148629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8a4cb-5c59-469e-bbec-c40f88726584 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:42.150730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1253/2510] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:57:41 2022] POST /identity/v3/users => generated 284 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:42.156089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3835e-4d1d-4bc5-a68f-543584261239 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.158150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-25b3835e-4d1d-4bc5-a68f-543584261239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:42.161958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1258/2511] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:57:42.168387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8a4cb-5c59-469e-bbec-c40f88726584 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.171996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3835e-4d1d-4bc5-a68f-543584261239 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:42.177720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8a24-53cb-4d83-854f-098e7b118552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:42.178366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8a24-53cb-4d83-854f-098e7b118552 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.178683 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8a24-53cb-4d83-854f-098e7b118552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.178991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8a24-53cb-4d83-854f-098e7b118552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.179633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a8a24-53cb-4d83-854f-098e7b118552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:42.180916 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:42.181901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a8a24-53cb-4d83-854f-098e7b118552 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:42.188750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8a4cb-5c59-469e-bbec-c40f88726584 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:42.193763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1254/2512] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:57:42.197016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfc3e9-50cf-4ac7-b805-f6445e3aa907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:42.198171 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8a24-53cb-4d83-854f-098e7b118552 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.198332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfc3e9-50cf-4ac7-b805-f6445e3aa907 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.198903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfc3e9-50cf-4ac7-b805-f6445e3aa907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.199495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfc3e9-50cf-4ac7-b805-f6445e3aa907 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.200788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfc3e9-50cf-4ac7-b805-f6445e3aa907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:42.204774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfc3e9-50cf-4ac7-b805-f6445e3aa907 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:42.208735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8a24-53cb-4d83-854f-098e7b118552 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:42.214646 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaef1e3-cfc6-4a44-a963-25b5940d4588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:42.215408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef1e3-cfc6-4a44-a963-25b5940d4588 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.215692 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef1e3-cfc6-4a44-a963-25b5940d4588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.215963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef1e3-cfc6-4a44-a963-25b5940d4588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.216680 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaef1e3-cfc6-4a44-a963-25b5940d4588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:42.218321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaef1e3-cfc6-4a44-a963-25b5940d4588 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:42.231574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1255/2513] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:57:42.235643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaef1e3-cfc6-4a44-a963-25b5940d4588 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.247489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaef1e3-cfc6-4a44-a963-25b5940d4588 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:42.255508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dbb56-9bab-443d-a15e-6c983745c8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:42.256246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dbb56-9bab-443d-a15e-6c983745c8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.256585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dbb56-9bab-443d-a15e-6c983745c8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.257027 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dbb56-9bab-443d-a15e-6c983745c8f1 admin admin] PATH_INFO: `/v3/projects/745dfeda1b7f4bf6a26a14161e0f44f5/users/2dcd829b766341e2a35f7235f526c51a/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.276541 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dbb56-9bab-443d-a15e-6c983745c8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=745dfeda1b7f4bf6a26a14161e0f44f5, user_id=2dcd829b766341e2a35f7235f526c51a, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:42.279400 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dbb56-9bab-443d-a15e-6c983745c8f1 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:42.324606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1259/2514] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:57:42 2022] POST /identity/v3/projects => generated 379 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:42.328786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfc3e9-50cf-4ac7-b805-f6445e3aa907 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.360265 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1256/2515] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:42 2022] PUT /identity/v3/projects/745dfeda1b7f4bf6a26a14161e0f44f5/users/2dcd829b766341e2a35f7235f526c51a/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:57:42.373410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7abcaa-e6ee-4264-aa2e-d0cbb97aedaa None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.373764 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7abcaa-e6ee-4264-aa2e-d0cbb97aedaa None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.374047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7abcaa-e6ee-4264-aa2e-d0cbb97aedaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.398548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfc3e9-50cf-4ac7-b805-f6445e3aa907 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:42.405598 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7c7abcaa-e6ee-4264-aa2e-d0cbb97aedaa None None] MFA Rules not processed for user `2dcd829b766341e2a35f7235f526c51a`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:42.407123 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7c7abcaa-e6ee-4264-aa2e-d0cbb97aedaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:42.461514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:42.470568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693befe-a955-4ee5-af16-7928226483eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:42.471185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693befe-a955-4ee5-af16-7928226483eb admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.471497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693befe-a955-4ee5-af16-7928226483eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.471774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693befe-a955-4ee5-af16-7928226483eb admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.472416 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693befe-a955-4ee5-af16-7928226483eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:42.474452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693befe-a955-4ee5-af16-7928226483eb admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:42.493278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1257/2516] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:57:42.510545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:42.513486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7abcaa-e6ee-4264-aa2e-d0cbb97aedaa None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.513486 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7c7abcaa-e6ee-4264-aa2e-d0cbb97aedaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:42.577400 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7abcaa-e6ee-4264-aa2e-d0cbb97aedaa None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:42.581205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a95b4-0978-42b1-88c0-f32c14ca1376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:42.581836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a95b4-0978-42b1-88c0-f32c14ca1376 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.582116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a95b4-0978-42b1-88c0-f32c14ca1376 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.582393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a95b4-0978-42b1-88c0-f32c14ca1376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.589800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1260/2517] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:57:42 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:42.592339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a95b4-0978-42b1-88c0-f32c14ca1376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:42.593792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a95b4-0978-42b1-88c0-f32c14ca1376 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:42.595543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693befe-a955-4ee5-af16-7928226483eb admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.605044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1258/2518] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:57:42.606279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693befe-a955-4ee5-af16-7928226483eb admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:42.611346 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2823ff-8be2-498e-9769-e0b35f920368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:42.611936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2823ff-8be2-498e-9769-e0b35f920368 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.612281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2823ff-8be2-498e-9769-e0b35f920368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.612471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2823ff-8be2-498e-9769-e0b35f920368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.613207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2823ff-8be2-498e-9769-e0b35f920368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:42.614726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2823ff-8be2-498e-9769-e0b35f920368 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:42.632572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1261/2519] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:57:42.638191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a95b4-0978-42b1-88c0-f32c14ca1376 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.654225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a95b4-0978-42b1-88c0-f32c14ca1376 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:42.663128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7cb38-869f-41ee-87a1-56b78acc6064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:42.664161 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7cb38-869f-41ee-87a1-56b78acc6064 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.664445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7cb38-869f-41ee-87a1-56b78acc6064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.664720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7cb38-869f-41ee-87a1-56b78acc6064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.665383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7cb38-869f-41ee-87a1-56b78acc6064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:42.667159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7cb38-869f-41ee-87a1-56b78acc6064 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:42.681847 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1259/2520] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:42.683552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2823ff-8be2-498e-9769-e0b35f920368 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.695073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2823ff-8be2-498e-9769-e0b35f920368 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:42.700930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63957f63-967b-4898-9d79-3db0baecffa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:42.701585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63957f63-967b-4898-9d79-3db0baecffa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.703839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63957f63-967b-4898-9d79-3db0baecffa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.703839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63957f63-967b-4898-9d79-3db0baecffa3 admin admin] PATH_INFO: `/v3/projects/c78ff4cbc0994af8855b1ac67cb281fe/users/1e63436f5691431d802706a8d5a6df7f/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.718700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63957f63-967b-4898-9d79-3db0baecffa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78ff4cbc0994af8855b1ac67cb281fe, user_id=1e63436f5691431d802706a8d5a6df7f, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:42.721653 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63957f63-967b-4898-9d79-3db0baecffa3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:42.859494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1262/2521] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:42 2022] PUT /identity/v3/projects/c78ff4cbc0994af8855b1ac67cb281fe/users/1e63436f5691431d802706a8d5a6df7f/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:57:42.864697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8ffba-1d54-4c0a-8948-70f0db6a4003 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:42.864984 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8ffba-1d54-4c0a-8948-70f0db6a4003 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:42.865255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8ffba-1d54-4c0a-8948-70f0db6a4003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:42.889591 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-cbe8ffba-1d54-4c0a-8948-70f0db6a4003 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:42.890879 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-cbe8ffba-1d54-4c0a-8948-70f0db6a4003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:42.953497 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1260/2522] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:57:42.963612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63957f63-967b-4898-9d79-3db0baecffa3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:42.965659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-63957f63-967b-4898-9d79-3db0baecffa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:43.026444 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63957f63-967b-4898-9d79-3db0baecffa3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:43.030560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0637c5f-cc03-4132-902c-80f7f68e7b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:43.031457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0637c5f-cc03-4132-902c-80f7f68e7b08 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:43.031949 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0637c5f-cc03-4132-902c-80f7f68e7b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:43.033206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0637c5f-cc03-4132-902c-80f7f68e7b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:43.050261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0637c5f-cc03-4132-902c-80f7f68e7b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:43.051752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0637c5f-cc03-4132-902c-80f7f68e7b08 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:43.062962 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1263/2523] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:57:46.573387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d055004-41fc-4816-ab65-d4f8685b0e0a None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:46.573959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d055004-41fc-4816-ab65-d4f8685b0e0a None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:46.574389 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d055004-41fc-4816-ab65-d4f8685b0e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:46.608854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-2d055004-41fc-4816-ab65-d4f8685b0e0a None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:46.610320 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2d055004-41fc-4816-ab65-d4f8685b0e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:46.627439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:46.683778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1261/2524] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:57:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:47.001822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0637c5f-cc03-4132-902c-80f7f68e7b08 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:47.003331 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a0637c5f-cc03-4132-902c-80f7f68e7b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:47.115901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0637c5f-cc03-4132-902c-80f7f68e7b08 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:47.124173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58d746-1710-4af0-91ff-290e8c6d7e9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:47.124853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58d746-1710-4af0-91ff-290e8c6d7e9b service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:47.125170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58d746-1710-4af0-91ff-290e8c6d7e9b service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:47.125449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58d746-1710-4af0-91ff-290e8c6d7e9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:47.138218 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58d746-1710-4af0-91ff-290e8c6d7e9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:47.140357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58d746-1710-4af0-91ff-290e8c6d7e9b service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:47.153807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1264/2525] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:57:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:57:51.049066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3d8c4-d951-4e71-aacd-d97df89a2869 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:51.049625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3d8c4-d951-4e71-aacd-d97df89a2869 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:51.049714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3d8c4-d951-4e71-aacd-d97df89a2869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:51.099008 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7ab3d8c4-d951-4e71-aacd-d97df89a2869 None None] MFA Rules not processed for user `1e63436f5691431d802706a8d5a6df7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:51.100245 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7ab3d8c4-d951-4e71-aacd-d97df89a2869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:51.209012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1262/2526] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:51.210234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58d746-1710-4af0-91ff-290e8c6d7e9b service swift] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:51.225184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58d746-1710-4af0-91ff-290e8c6d7e9b service swift] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:51.230935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edfc3b-acbc-4da8-a18c-eaeab006a3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:51.231656 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edfc3b-acbc-4da8-a18c-eaeab006a3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:51.232187 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edfc3b-acbc-4da8-a18c-eaeab006a3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:51.232807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edfc3b-acbc-4da8-a18c-eaeab006a3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:51.233850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3edfc3b-acbc-4da8-a18c-eaeab006a3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:51.236064 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3edfc3b-acbc-4da8-a18c-eaeab006a3f6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:51.368238 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3d8c4-d951-4e71-aacd-d97df89a2869 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:51.370933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1265/2527] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:57:51 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:51.385075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3d8c4-d951-4e71-aacd-d97df89a2869 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:51.484802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d3f80-777e-4427-8712-d9f58d1bd61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:51.485799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d3f80-777e-4427-8712-d9f58d1bd61f admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:51.486301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d3f80-777e-4427-8712-d9f58d1bd61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:51.486851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d3f80-777e-4427-8712-d9f58d1bd61f admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:51.488141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d3f80-777e-4427-8712-d9f58d1bd61f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:51.492397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d3f80-777e-4427-8712-d9f58d1bd61f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:51.585942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1263/2528] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:57:51 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:51.591619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edfc3b-acbc-4da8-a18c-eaeab006a3f6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:51.605069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edfc3b-acbc-4da8-a18c-eaeab006a3f6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:51.611509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014ad3-0cf6-471e-a0ae-f206f7143363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:51.611509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014ad3-0cf6-471e-a0ae-f206f7143363 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:51.613318 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014ad3-0cf6-471e-a0ae-f206f7143363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:51.613723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014ad3-0cf6-471e-a0ae-f206f7143363 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:51.614557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19014ad3-0cf6-471e-a0ae-f206f7143363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:51.616482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19014ad3-0cf6-471e-a0ae-f206f7143363 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:51.631206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1266/2529] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:51.636342 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d3f80-777e-4427-8712-d9f58d1bd61f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:51.668650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d3f80-777e-4427-8712-d9f58d1bd61f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:51.668650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f24e29-e6a1-49b6-9834-3ddce86dc780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:51.668650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f24e29-e6a1-49b6-9834-3ddce86dc780 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:51.668650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f24e29-e6a1-49b6-9834-3ddce86dc780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:51.669078 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f24e29-e6a1-49b6-9834-3ddce86dc780 admin admin] PATH_INFO: `/v3/projects/6a86a9923ac94e39b93c5f2107a1f3cf/users/8ea434be0ec14904809ba7b48426a372/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:51.674853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f24e29-e6a1-49b6-9834-3ddce86dc780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a86a9923ac94e39b93c5f2107a1f3cf, user_id=8ea434be0ec14904809ba7b48426a372, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:51.677583 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f24e29-e6a1-49b6-9834-3ddce86dc780 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:52.212532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1264/2530] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:51 2022] PUT /identity/v3/projects/6a86a9923ac94e39b93c5f2107a1f3cf/users/8ea434be0ec14904809ba7b48426a372/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:52.218729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014ad3-0cf6-471e-a0ae-f206f7143363 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:52.231352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014ad3-0cf6-471e-a0ae-f206f7143363 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:52.237008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:52.331864 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64d22d-3540-42a0-a0be-de0b14af86c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:52.332520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64d22d-3540-42a0-a0be-de0b14af86c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:52.332799 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64d22d-3540-42a0-a0be-de0b14af86c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:52.333072 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64d22d-3540-42a0-a0be-de0b14af86c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:52.333727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64d22d-3540-42a0-a0be-de0b14af86c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:52.335621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64d22d-3540-42a0-a0be-de0b14af86c0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:52.359346 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1267/2531] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:57:52.364073 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f24e29-e6a1-49b6-9834-3ddce86dc780 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:52.394558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f24e29-e6a1-49b6-9834-3ddce86dc780 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:52.411970 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe88be1-babf-44e2-aca8-5a32a001e016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:52.412804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe88be1-babf-44e2-aca8-5a32a001e016 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:52.413092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe88be1-babf-44e2-aca8-5a32a001e016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:52.413371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe88be1-babf-44e2-aca8-5a32a001e016 admin admin] PATH_INFO: `/v3/projects/6a86a9923ac94e39b93c5f2107a1f3cf/users/8ea434be0ec14904809ba7b48426a372/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:52.419790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe88be1-babf-44e2-aca8-5a32a001e016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a86a9923ac94e39b93c5f2107a1f3cf, user_id=8ea434be0ec14904809ba7b48426a372, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:52.424574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe88be1-babf-44e2-aca8-5a32a001e016 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:52.453984 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1265/2532] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:52 2022] PUT /identity/v3/projects/6a86a9923ac94e39b93c5f2107a1f3cf/users/8ea434be0ec14904809ba7b48426a372/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:54.864434 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64d22d-3540-42a0-a0be-de0b14af86c0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:54.889109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64d22d-3540-42a0-a0be-de0b14af86c0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:55.040658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:55.043115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:55.044113 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:55.044884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] PATH_INFO: `/v3/users/2dcd829b766341e2a35f7235f526c51a` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:55.049199 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dcd829b766341e2a35f7235f526c51a)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:55.054688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:55.294152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dcd829b766341e2a35f7235f526c51a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:55.329630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dcd829b766341e2a35f7235f526c51a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:55.351423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dcd829b766341e2a35f7235f526c51a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:55.441078 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dcd829b766341e2a35f7235f526c51a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:55.441974 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dcd829b766341e2a35f7235f526c51a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:55.455234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1268/2533] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:57:54 2022] DELETE /identity/v3/users/2dcd829b766341e2a35f7235f526c51a => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:55.459779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37f46d-a7e4-4855-ba85-7f21506078de None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:55.460343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37f46d-a7e4-4855-ba85-7f21506078de None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:55.460967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37f46d-a7e4-4855-ba85-7f21506078de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:55.475833 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:55.507600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-2b37f46d-a7e4-4855-ba85-7f21506078de None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:55.511858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2b37f46d-a7e4-4855-ba85-7f21506078de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:55.683795 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1266/2534] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:55.690624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:55.693344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:55.740540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:55.824688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ea709-5a97-41a9-84dc-ec2e069fed23 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:55.834160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3417ac4-4a59-4e85-815d-c491e99f0a4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:55.835102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3417ac4-4a59-4e85-815d-c491e99f0a4a service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:55.835734 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3417ac4-4a59-4e85-815d-c491e99f0a4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:55.836002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3417ac4-4a59-4e85-815d-c491e99f0a4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:55.849502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3417ac4-4a59-4e85-815d-c491e99f0a4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:55.851329 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3417ac4-4a59-4e85-815d-c491e99f0a4a service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:55.874561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1269/2535] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:57:57.051468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37f46d-a7e4-4855-ba85-7f21506078de None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:57.051468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2b37f46d-a7e4-4855-ba85-7f21506078de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:57.075021 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37f46d-a7e4-4855-ba85-7f21506078de None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:57.098906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:57.115353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:57.115648 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:57.115925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] PATH_INFO: `/v3/projects/745dfeda1b7f4bf6a26a14161e0f44f5` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:57.123921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] RBAC: Authorizing `identity:delete_project(project_id=745dfeda1b7f4bf6a26a14161e0f44f5)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:57.129536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:57.279765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '745dfeda1b7f4bf6a26a14161e0f44f5', 'request_id': 'req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:57.389709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '745dfeda1b7f4bf6a26a14161e0f44f5', 'request_id': 'req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:57.399811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '745dfeda1b7f4bf6a26a14161e0f44f5', 'request_id': 'req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:57.417660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] The token cache is being invalidate because project 745dfeda1b7f4bf6a26a14161e0f44f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:57:57.418341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:57.419291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-bd59db3b-a27c-4d1a-ba29-97bacd9b82be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:57:57.421370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1267/2536] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:57:57 2022] DELETE /identity/v3/projects/745dfeda1b7f4bf6a26a14161e0f44f5 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:57.426341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcff8d-bc93-48ff-ad2d-05bf78b50b6c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:57.429827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcff8d-bc93-48ff-ad2d-05bf78b50b6c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:57.429827 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcff8d-bc93-48ff-ad2d-05bf78b50b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:57.455231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d6dcff8d-bc93-48ff-ad2d-05bf78b50b6c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:57.469090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d6dcff8d-bc93-48ff-ad2d-05bf78b50b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:57.558251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1270/2537] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:57.566130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39242a-24be-4c39-9da0-533e26483514 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:57.566655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39242a-24be-4c39-9da0-533e26483514 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:57.567039 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39242a-24be-4c39-9da0-533e26483514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:57.617128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-ec39242a-24be-4c39-9da0-533e26483514 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:57.618656 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ec39242a-24be-4c39-9da0-533e26483514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:57.651094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1268/2538] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:57.658269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcff8d-bc93-48ff-ad2d-05bf78b50b6c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:57.681183 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcff8d-bc93-48ff-ad2d-05bf78b50b6c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:57.703758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4f122-3cf8-4ba9-aa4b-9d355a6c0a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:57.705789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4f122-3cf8-4ba9-aa4b-9d355a6c0a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:57.708548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4f122-3cf8-4ba9-aa4b-9d355a6c0a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:57.708548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4f122-3cf8-4ba9-aa4b-9d355a6c0a00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:57.708548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4f122-3cf8-4ba9-aa4b-9d355a6c0a00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:57:57.708548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4f122-3cf8-4ba9-aa4b-9d355a6c0a00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:57.715809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4f122-3cf8-4ba9-aa4b-9d355a6c0a00 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:57.734327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1271/2539] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:57:57.744141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39242a-24be-4c39-9da0-533e26483514 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:57.757887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39242a-24be-4c39-9da0-533e26483514 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:57.765549 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a8c1c-354e-48b4-be9b-dd86284e5a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:57.768097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a8c1c-354e-48b4-be9b-dd86284e5a6f admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:57.768781 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a8c1c-354e-48b4-be9b-dd86284e5a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:57.769234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a8c1c-354e-48b4-be9b-dd86284e5a6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:57.770087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a8c1c-354e-48b4-be9b-dd86284e5a6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:57.772663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a8c1c-354e-48b4-be9b-dd86284e5a6f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:57.917495 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1269/2540] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:57:57 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:57:57.921015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4f122-3cf8-4ba9-aa4b-9d355a6c0a00 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:57.936804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4f122-3cf8-4ba9-aa4b-9d355a6c0a00 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:58.147879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d158b52-6b41-40f0-92a9-8432e2eb878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:58.147879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d158b52-6b41-40f0-92a9-8432e2eb878b admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:58.147879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d158b52-6b41-40f0-92a9-8432e2eb878b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:58.151302 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d158b52-6b41-40f0-92a9-8432e2eb878b admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:58.152404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d158b52-6b41-40f0-92a9-8432e2eb878b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:58.155512 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d158b52-6b41-40f0-92a9-8432e2eb878b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:58.349616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1272/2541] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:57:57 2022] POST /identity/v3/users => generated 283 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:58.354744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a8c1c-354e-48b4-be9b-dd86284e5a6f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:58.371449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a8c1c-354e-48b4-be9b-dd86284e5a6f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:58.385679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac6f20-1fa2-4bec-abb2-20dcfcae9275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:58.387644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac6f20-1fa2-4bec-abb2-20dcfcae9275 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:58.390650 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac6f20-1fa2-4bec-abb2-20dcfcae9275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:58.391198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac6f20-1fa2-4bec-abb2-20dcfcae9275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:58.393903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac6f20-1fa2-4bec-abb2-20dcfcae9275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:58.398304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac6f20-1fa2-4bec-abb2-20dcfcae9275 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:58.429446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1270/2542] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:58.432154 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d158b52-6b41-40f0-92a9-8432e2eb878b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:58.450619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d158b52-6b41-40f0-92a9-8432e2eb878b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:58.457648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35eb6a1-daec-400d-851f-95cd7ce886b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:58.458878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eb6a1-daec-400d-851f-95cd7ce886b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:58.459400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eb6a1-daec-400d-851f-95cd7ce886b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:58.459884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eb6a1-daec-400d-851f-95cd7ce886b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:58.461645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35eb6a1-daec-400d-851f-95cd7ce886b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:58.464820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35eb6a1-daec-400d-851f-95cd7ce886b5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:58.479562 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1273/2543] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:58.483344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac6f20-1fa2-4bec-abb2-20dcfcae9275 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:58.503527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac6f20-1fa2-4bec-abb2-20dcfcae9275 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:58.514602 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db8612-32a8-4bc8-8c3f-f2feb17254f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:58.515442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db8612-32a8-4bc8-8c3f-f2feb17254f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:58.515825 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db8612-32a8-4bc8-8c3f-f2feb17254f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:58.516216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db8612-32a8-4bc8-8c3f-f2feb17254f3 admin admin] PATH_INFO: `/v3/projects/71a4e04b99e547de83c1e2359f5ce1fa/users/d80c63ef5d3848749ae96c57c36488e0/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:58.558042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db8612-32a8-4bc8-8c3f-f2feb17254f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a4e04b99e547de83c1e2359f5ce1fa, user_id=d80c63ef5d3848749ae96c57c36488e0, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:58.561069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db8612-32a8-4bc8-8c3f-f2feb17254f3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:58.712882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1271/2544] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:58 2022] PUT /identity/v3/projects/71a4e04b99e547de83c1e2359f5ce1fa/users/d80c63ef5d3848749ae96c57c36488e0/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:58.715577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba02734-78c2-4a88-b5c9-b026e21668e8 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:58.716174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba02734-78c2-4a88-b5c9-b026e21668e8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:58.716692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba02734-78c2-4a88-b5c9-b026e21668e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:58.829103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6ba02734-78c2-4a88-b5c9-b026e21668e8 None None] MFA Rules not processed for user `d80c63ef5d3848749ae96c57c36488e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:57:58.836597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6ba02734-78c2-4a88-b5c9-b026e21668e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:57:58.898791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:59.026108 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1274/2545] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:57:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:57:59.030249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db8612-32a8-4bc8-8c3f-f2feb17254f3 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:59.080991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:57:59.201664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db8612-32a8-4bc8-8c3f-f2feb17254f3 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:59.230855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4eec20-21c3-44d0-87e2-561179192848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:59.231793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4eec20-21c3-44d0-87e2-561179192848 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:59.234046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4eec20-21c3-44d0-87e2-561179192848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:59.234459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4eec20-21c3-44d0-87e2-561179192848 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:59.235207 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4eec20-21c3-44d0-87e2-561179192848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:59.237617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4eec20-21c3-44d0-87e2-561179192848 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:59.412120 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba02734-78c2-4a88-b5c9-b026e21668e8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:59.413119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1272/2546] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:57:59 2022] POST /identity/v3/projects => generated 369 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:59.457395 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba02734-78c2-4a88-b5c9-b026e21668e8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:59.608570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdecd6-b387-46ac-9257-068c0ebef295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:59.609206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdecd6-b387-46ac-9257-068c0ebef295 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:59.609487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdecd6-b387-46ac-9257-068c0ebef295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:59.609763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdecd6-b387-46ac-9257-068c0ebef295 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:59.610433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdecd6-b387-46ac-9257-068c0ebef295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:59.612440 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdecd6-b387-46ac-9257-068c0ebef295 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:59.708119 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1275/2547] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:57:59 2022] POST /identity/v3/users => generated 291 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:57:59.710532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4eec20-21c3-44d0-87e2-561179192848 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:59.729410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4eec20-21c3-44d0-87e2-561179192848 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:59.738948 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00543b-e5f0-4a34-8023-34419571ff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:59.739903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00543b-e5f0-4a34-8023-34419571ff5c admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:59.740333 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00543b-e5f0-4a34-8023-34419571ff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:59.740778 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00543b-e5f0-4a34-8023-34419571ff5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:59.742206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00543b-e5f0-4a34-8023-34419571ff5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:59.744965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00543b-e5f0-4a34-8023-34419571ff5c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:59.761093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1273/2548] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:57:59.766810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdecd6-b387-46ac-9257-068c0ebef295 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:59.781955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdecd6-b387-46ac-9257-068c0ebef295 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:57:59.786797 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a7a44-ffdc-4eb2-a5c9-23401c65db00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:57:59.788116 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a7a44-ffdc-4eb2-a5c9-23401c65db00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:57:59.788116 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a7a44-ffdc-4eb2-a5c9-23401c65db00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:57:59.788116 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a7a44-ffdc-4eb2-a5c9-23401c65db00 admin admin] PATH_INFO: `/v3/projects/0f0421576b8c4c71be67ff4bacbfb35c/users/ec241162ccf0462488c962fa4dc8375d/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:57:59.813432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a7a44-ffdc-4eb2-a5c9-23401c65db00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0421576b8c4c71be67ff4bacbfb35c, user_id=ec241162ccf0462488c962fa4dc8375d, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:57:59.818999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a7a44-ffdc-4eb2-a5c9-23401c65db00 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:57:59.955844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1276/2549] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:57:59 2022] PUT /identity/v3/projects/0f0421576b8c4c71be67ff4bacbfb35c/users/ec241162ccf0462488c962fa4dc8375d/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:57:59.959813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00543b-e5f0-4a34-8023-34419571ff5c admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:57:59.976060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00543b-e5f0-4a34-8023-34419571ff5c admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:00.094339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a3175-256b-43d1-b199-53cf53dfed8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:00.097531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a3175-256b-43d1-b199-53cf53dfed8e admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:00.097531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a3175-256b-43d1-b199-53cf53dfed8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:00.097531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a3175-256b-43d1-b199-53cf53dfed8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:00.097531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a3175-256b-43d1-b199-53cf53dfed8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:00.098273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a3175-256b-43d1-b199-53cf53dfed8e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:00.126203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1274/2550] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:58:00.130343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a7a44-ffdc-4eb2-a5c9-23401c65db00 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:00.147935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a7a44-ffdc-4eb2-a5c9-23401c65db00 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:00.164452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6caf7-8d42-4f3b-95ab-f5a9e1e3e8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:00.170459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6caf7-8d42-4f3b-95ab-f5a9e1e3e8e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:00.170459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6caf7-8d42-4f3b-95ab-f5a9e1e3e8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:00.170459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6caf7-8d42-4f3b-95ab-f5a9e1e3e8e1 admin admin] PATH_INFO: `/v3/projects/0f0421576b8c4c71be67ff4bacbfb35c/users/ec241162ccf0462488c962fa4dc8375d/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:00.176710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6caf7-8d42-4f3b-95ab-f5a9e1e3e8e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0421576b8c4c71be67ff4bacbfb35c, user_id=ec241162ccf0462488c962fa4dc8375d, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:00.176710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d6caf7-8d42-4f3b-95ab-f5a9e1e3e8e1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:00.207013 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1277/2551] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:58:00 2022] PUT /identity/v3/projects/0f0421576b8c4c71be67ff4bacbfb35c/users/ec241162ccf0462488c962fa4dc8375d/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:58:00.219962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21eb7-117c-4966-bfaf-ad7961d2bcc6 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:00.220419 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21eb7-117c-4966-bfaf-ad7961d2bcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:00.220806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21eb7-117c-4966-bfaf-ad7961d2bcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:00.286062 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-72d21eb7-117c-4966-bfaf-ad7961d2bcc6 None None] MFA Rules not processed for user `ec241162ccf0462488c962fa4dc8375d`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:00.287560 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-72d21eb7-117c-4966-bfaf-ad7961d2bcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:00.438257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1275/2552] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:58:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:00.452093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6caf7-8d42-4f3b-95ab-f5a9e1e3e8e1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:00.453880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c5d6caf7-8d42-4f3b-95ab-f5a9e1e3e8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:00.573547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6caf7-8d42-4f3b-95ab-f5a9e1e3e8e1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:00.589720 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090732c-0e9d-47b1-bb76-c7ab1fde5fb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:00.592761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090732c-0e9d-47b1-bb76-c7ab1fde5fb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:00.593204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090732c-0e9d-47b1-bb76-c7ab1fde5fb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:00.595574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090732c-0e9d-47b1-bb76-c7ab1fde5fb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:00.612152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090732c-0e9d-47b1-bb76-c7ab1fde5fb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:00.613673 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090732c-0e9d-47b1-bb76-c7ab1fde5fb2 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:00.636186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1278/2553] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:58:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:58:01.029773 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d21eb7-117c-4966-bfaf-ad7961d2bcc6 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:01.060439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d21eb7-117c-4966-bfaf-ad7961d2bcc6 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:01.068011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5feae2-d8c8-4e53-b93b-c242fa6f9925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:01.068715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5feae2-d8c8-4e53-b93b-c242fa6f9925 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:01.069047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5feae2-d8c8-4e53-b93b-c242fa6f9925 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:01.069324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5feae2-d8c8-4e53-b93b-c242fa6f9925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:01.100140 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:01.166769 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5feae2-d8c8-4e53-b93b-c242fa6f9925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:01.168316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5feae2-d8c8-4e53-b93b-c242fa6f9925 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:01.185414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1276/2554] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:58:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:58:02.346659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c1a62-b276-4412-9f05-b75193730943 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:02.346659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c1a62-b276-4412-9f05-b75193730943 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:02.346659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c1a62-b276-4412-9f05-b75193730943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:02.346659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-964c1a62-b276-4412-9f05-b75193730943 None None] MFA Rules not processed for user `8ea434be0ec14904809ba7b48426a372`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:02.346659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-964c1a62-b276-4412-9f05-b75193730943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:02.346659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:02.348899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1279/2555] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:58:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:02.349487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5feae2-d8c8-4e53-b93b-c242fa6f9925 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:02.349487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3d5feae2-d8c8-4e53-b93b-c242fa6f9925 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:02.441138 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5feae2-d8c8-4e53-b93b-c242fa6f9925 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:02.446993 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c756e9-a3da-40d1-9c63-597b3666d804 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:02.447731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c756e9-a3da-40d1-9c63-597b3666d804 service nova] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:02.448112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c756e9-a3da-40d1-9c63-597b3666d804 service nova] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:02.448438 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c756e9-a3da-40d1-9c63-597b3666d804 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:02.450584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c2c756e9-a3da-40d1-9c63-597b3666d804 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:02.517645 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c756e9-a3da-40d1-9c63-597b3666d804 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:02.519478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c756e9-a3da-40d1-9c63-597b3666d804 service nova] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:02.529098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1277/2556] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:58:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:58:12.524528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=75 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:12.697984 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=26 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:13.224255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa07f94-1ed0-4cc3-918d-8d04f1c1d3f2 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:13.225493 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa07f94-1ed0-4cc3-918d-8d04f1c1d3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:13.225961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa07f94-1ed0-4cc3-918d-8d04f1c1d3f2 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:13.230490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1280/2557] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:58:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:58:13.236826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598466-39e7-45e7-9058-5b96b9554fd3 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:13.237111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598466-39e7-45e7-9058-5b96b9554fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:13.237686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598466-39e7-45e7-9058-5b96b9554fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:13.264222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-04598466-39e7-45e7-9058-5b96b9554fd3 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:13.264222 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-04598466-39e7-45e7-9058-5b96b9554fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:13.272418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1278/2558] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:58:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:13.292141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa07f94-1ed0-4cc3-918d-8d04f1c1d3f2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:13.292141 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9aa07f94-1ed0-4cc3-918d-8d04f1c1d3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:13.367549 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa07f94-1ed0-4cc3-918d-8d04f1c1d3f2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:13.372713 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70a2df-4263-40ff-84b4-0b2e1673872f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a59337d20e4678b8a16ac35bffbdb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:13.373323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70a2df-4263-40ff-84b4-0b2e1673872f service glance] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:13.373602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70a2df-4263-40ff-84b4-0b2e1673872f service glance] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:13.373875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70a2df-4263-40ff-84b4-0b2e1673872f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:13.386648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70a2df-4263-40ff-84b4-0b2e1673872f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:13.388501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70a2df-4263-40ff-84b4-0b2e1673872f service glance] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:13.401629 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1281/2559] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:58:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:58:13.613967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7278cf3-47ea-4692-a2cf-bafeb236562e None None] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:13.614530 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7278cf3-47ea-4692-a2cf-bafeb236562e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:13.614671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7278cf3-47ea-4692-a2cf-bafeb236562e None None] PATH_INFO: `/` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:13.616011 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1279/2560] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:58:13.623527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87647fb6-0c4e-45f0-ad56-2ed0263035a1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:13.623857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87647fb6-0c4e-45f0-ad56-2ed0263035a1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:13.624139 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87647fb6-0c4e-45f0-ad56-2ed0263035a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:13.648002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-87647fb6-0c4e-45f0-ad56-2ed0263035a1 None None] MFA Rules not processed for user `863bcfa6b8694182b9d163133a141907`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:13.651051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-87647fb6-0c4e-45f0-ad56-2ed0263035a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:13.710511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1282/2561] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:58:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:13.723574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7278cf3-47ea-4692-a2cf-bafeb236562e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:13.725325 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b7278cf3-47ea-4692-a2cf-bafeb236562e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:13.743475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7278cf3-47ea-4692-a2cf-bafeb236562e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:13.747900 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12573081-c7bd-4eba-ad9f-befa9f6178a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:13.748609 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573081-c7bd-4eba-ad9f-befa9f6178a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:13.748893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573081-c7bd-4eba-ad9f-befa9f6178a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:13.749168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573081-c7bd-4eba-ad9f-befa9f6178a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:13.757275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12573081-c7bd-4eba-ad9f-befa9f6178a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:13.758761 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12573081-c7bd-4eba-ad9f-befa9f6178a4 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:13.767089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1280/2562] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:58:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:58:13.883463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8a5c7-ed40-4438-b857-efe39ce5b729 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:13.884360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8a5c7-ed40-4438-b857-efe39ce5b729 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:13.884893 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8a5c7-ed40-4438-b857-efe39ce5b729 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:13.886531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1283/2563] 10.222.0.58 () {54 vars in 921 bytes} [Mon Aug 29 23:58:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 23:58:13.894752 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e7bb-8105-4ec5-80bd-330abf957e68 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:13.895252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e7bb-8105-4ec5-80bd-330abf957e68 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:13.897561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e7bb-8105-4ec5-80bd-330abf957e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:13.933151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-e0f5e7bb-8105-4ec5-80bd-330abf957e68 None None] MFA Rules not processed for user `e4a8261d92064b838fc6e8f415879566`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:13.934746 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-e0f5e7bb-8105-4ec5-80bd-330abf957e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:13.944631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1281/2564] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Aug 29 23:58:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:13.955936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8a5c7-ed40-4438-b857-efe39ce5b729 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:13.968397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8a5c7-ed40-4438-b857-efe39ce5b729 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:13.973873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13380b-c0d7-4bfa-b015-2cefb3c26ee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:13.975571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13380b-c0d7-4bfa-b015-2cefb3c26ee9 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:13.975928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13380b-c0d7-4bfa-b015-2cefb3c26ee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:13.984815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13380b-c0d7-4bfa-b015-2cefb3c26ee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:13.992402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13380b-c0d7-4bfa-b015-2cefb3c26ee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:13.994958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13380b-c0d7-4bfa-b015-2cefb3c26ee9 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:14.007078 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1284/2565] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:58:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:58:17.452284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3a83-f3e1-475d-84e9-acc1dc494c9d None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:17.458632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3a83-f3e1-475d-84e9-acc1dc494c9d None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:17.458632 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3a83-f3e1-475d-84e9-acc1dc494c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:17.490708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d5cb3a83-f3e1-475d-84e9-acc1dc494c9d None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:17.491817 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d5cb3a83-f3e1-475d-84e9-acc1dc494c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:17.564537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1282/2566] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Aug 29 23:58:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:17.583360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13380b-c0d7-4bfa-b015-2cefb3c26ee9 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:17.614606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ae13380b-c0d7-4bfa-b015-2cefb3c26ee9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:17.675977 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13380b-c0d7-4bfa-b015-2cefb3c26ee9 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:17.683033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c462778-eb8d-4142-a1fe-8abd71ae031d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:17.683957 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c462778-eb8d-4142-a1fe-8abd71ae031d service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:17.684519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c462778-eb8d-4142-a1fe-8abd71ae031d service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:17.685171 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c462778-eb8d-4142-a1fe-8abd71ae031d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:17.697928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c462778-eb8d-4142-a1fe-8abd71ae031d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:17.700170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c462778-eb8d-4142-a1fe-8abd71ae031d service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:17.711474 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1285/2567] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:58:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:58:26.529173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32a8f3-f9b5-4045-b57e-e104be72725c None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:26.529173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32a8f3-f9b5-4045-b57e-e104be72725c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:26.529173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32a8f3-f9b5-4045-b57e-e104be72725c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:26.529173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-bb32a8f3-f9b5-4045-b57e-e104be72725c None None] MFA Rules not processed for user `1e63436f5691431d802706a8d5a6df7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:26.529173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bb32a8f3-f9b5-4045-b57e-e104be72725c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:26.531881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1283/2568] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:58:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:27.707216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=87 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:36.516279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=40 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:38.737253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c70310-c3a1-4917-abc5-686f44ea6529 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:38.737253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c70310-c3a1-4917-abc5-686f44ea6529 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:38.737253 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c70310-c3a1-4917-abc5-686f44ea6529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:38.788121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-18c70310-c3a1-4917-abc5-686f44ea6529 None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:38.788121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-18c70310-c3a1-4917-abc5-686f44ea6529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:38.812601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1286/2569] 10.222.0.58 () {58 vars in 967 bytes} [Mon Aug 29 23:58:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:38.833907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32a8f3-f9b5-4045-b57e-e104be72725c None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:38.858020 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32a8f3-f9b5-4045-b57e-e104be72725c None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:38.872798 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07de68-3530-4d96-8fd8-5ee07b142f55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:38.873656 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07de68-3530-4d96-8fd8-5ee07b142f55 service swift] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:38.873936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07de68-3530-4d96-8fd8-5ee07b142f55 service swift] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:38.874381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07de68-3530-4d96-8fd8-5ee07b142f55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:38.904536 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07de68-3530-4d96-8fd8-5ee07b142f55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:38.906975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07de68-3530-4d96-8fd8-5ee07b142f55 service swift] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:38.922934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1284/2570] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Aug 29 23:58:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:58:39.246126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c70310-c3a1-4917-abc5-686f44ea6529 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:39.319243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c70310-c3a1-4917-abc5-686f44ea6529 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:39.381232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c4f39-f45c-423f-82ae-266243abcd97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:39.385771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c4f39-f45c-423f-82ae-266243abcd97 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:39.386539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c4f39-f45c-423f-82ae-266243abcd97 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:39.387602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c4f39-f45c-423f-82ae-266243abcd97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:39.393166 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-da4c4f39-f45c-423f-82ae-266243abcd97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:39.449411 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c4f39-f45c-423f-82ae-266243abcd97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:39.451157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c4f39-f45c-423f-82ae-266243abcd97 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:39.484961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1287/2571] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:58:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:58:46.290366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07de68-3530-4d96-8fd8-5ee07b142f55 service swift] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:46.292808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3f07de68-3530-4d96-8fd8-5ee07b142f55 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:46.308866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07de68-3530-4d96-8fd8-5ee07b142f55 service swift] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:46.312470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc31ed7-8988-4e3a-b8ff-452c1dd3e709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:46.313119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc31ed7-8988-4e3a-b8ff-452c1dd3e709 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:46.313357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc31ed7-8988-4e3a-b8ff-452c1dd3e709 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:46.313625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc31ed7-8988-4e3a-b8ff-452c1dd3e709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:46.315413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-dcc31ed7-8988-4e3a-b8ff-452c1dd3e709 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:46.388202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc31ed7-8988-4e3a-b8ff-452c1dd3e709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:46.389733 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc31ed7-8988-4e3a-b8ff-452c1dd3e709 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:46.410299 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1285/2572] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:58:49.461881 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=9 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:50.924058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c4f39-f45c-423f-82ae-266243abcd97 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:50.927176 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-da4c4f39-f45c-423f-82ae-266243abcd97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:50.959880 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c4f39-f45c-423f-82ae-266243abcd97 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:50.965701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6f62e-f02f-4625-8878-941d7b476a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:50.966340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6f62e-f02f-4625-8878-941d7b476a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:50.966649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6f62e-f02f-4625-8878-941d7b476a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:50.966920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6f62e-f02f-4625-8878-941d7b476a69 admin admin] PATH_INFO: `/v3/users/34ef7ee27c3948ac9df1ff94ce24618e` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:50.968073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6f62e-f02f-4625-8878-941d7b476a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34ef7ee27c3948ac9df1ff94ce24618e)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:50.970463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6f62e-f02f-4625-8878-941d7b476a69 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:51.212719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a7c6f62e-f02f-4625-8878-941d7b476a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34ef7ee27c3948ac9df1ff94ce24618e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:51.227238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a7c6f62e-f02f-4625-8878-941d7b476a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34ef7ee27c3948ac9df1ff94ce24618e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:51.235502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a7c6f62e-f02f-4625-8878-941d7b476a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34ef7ee27c3948ac9df1ff94ce24618e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:51.304059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a7c6f62e-f02f-4625-8878-941d7b476a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34ef7ee27c3948ac9df1ff94ce24618e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:51.304708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a7c6f62e-f02f-4625-8878-941d7b476a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34ef7ee27c3948ac9df1ff94ce24618e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:51.312101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1288/2573] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:58:50 2022] DELETE /identity/v3/users/34ef7ee27c3948ac9df1ff94ce24618e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:58:51.557834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc31ed7-8988-4e3a-b8ff-452c1dd3e709 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:51.570254 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc31ed7-8988-4e3a-b8ff-452c1dd3e709 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:51.628793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:51.629420 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:51.629712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:51.629980 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] PATH_INFO: `/v3/projects/80d82515633f4c2faa34d3225a6f26a6` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:51.631112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d82515633f4c2faa34d3225a6f26a6)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:51.633041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:51.816107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d82515633f4c2faa34d3225a6f26a6', 'request_id': 'req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:51.949570 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d82515633f4c2faa34d3225a6f26a6', 'request_id': 'req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:51.956765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d82515633f4c2faa34d3225a6f26a6', 'request_id': 'req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:51.969590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] The token cache is being invalidate because project 80d82515633f4c2faa34d3225a6f26a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:58:51.970060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:51.970755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-60780d91-00d7-4cb3-bc4d-b44674bdf5d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:51.972677 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1286/2574] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:58:51 2022] DELETE /identity/v3/projects/80d82515633f4c2faa34d3225a6f26a6 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:58:51.977987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def4012-ba9b-444f-ba62-0f03f4a5af9d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:51.978511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def4012-ba9b-444f-ba62-0f03f4a5af9d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:51.978630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def4012-ba9b-444f-ba62-0f03f4a5af9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:51.999688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2def4012-ba9b-444f-ba62-0f03f4a5af9d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:52.000932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2def4012-ba9b-444f-ba62-0f03f4a5af9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:52.063581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1289/2575] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:52.071844 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e260a8-0b09-4f3f-850e-2e87d71040a6 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:52.072124 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e260a8-0b09-4f3f-850e-2e87d71040a6 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:52.072393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e260a8-0b09-4f3f-850e-2e87d71040a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:52.095741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f6e260a8-0b09-4f3f-850e-2e87d71040a6 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:52.097806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f6e260a8-0b09-4f3f-850e-2e87d71040a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:52.111102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1287/2576] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:58:52.112727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def4012-ba9b-444f-ba62-0f03f4a5af9d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:52.125613 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def4012-ba9b-444f-ba62-0f03f4a5af9d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:52.133048 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3c88a-e9cc-47fc-94f8-bee44f50b38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:52.134002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3c88a-e9cc-47fc-94f8-bee44f50b38f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:52.134286 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3c88a-e9cc-47fc-94f8-bee44f50b38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:52.134604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3c88a-e9cc-47fc-94f8-bee44f50b38f admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:52.135314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3c88a-e9cc-47fc-94f8-bee44f50b38f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:58:52.136238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3c88a-e9cc-47fc-94f8-bee44f50b38f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:52.138196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3c88a-e9cc-47fc-94f8-bee44f50b38f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:52.152506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1290/2577] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:58:52.156930 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e260a8-0b09-4f3f-850e-2e87d71040a6 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:52.164664 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:52.169857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e260a8-0b09-4f3f-850e-2e87d71040a6 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:52.175348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce40f5-2b2f-4344-b502-9388842453a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:52.176064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce40f5-2b2f-4344-b502-9388842453a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:52.176416 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce40f5-2b2f-4344-b502-9388842453a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:52.176673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce40f5-2b2f-4344-b502-9388842453a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:52.177450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce40f5-2b2f-4344-b502-9388842453a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:52.179539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce40f5-2b2f-4344-b502-9388842453a9 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:52.325137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1288/2578] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:58:52 2022] POST /identity/v3/projects => generated 403 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:58:52.329710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3c88a-e9cc-47fc-94f8-bee44f50b38f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:52.340029 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3c88a-e9cc-47fc-94f8-bee44f50b38f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:52.411199 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f0089-162b-4edd-bede-58f049f5665b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:52.411821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f0089-162b-4edd-bede-58f049f5665b admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:52.412128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f0089-162b-4edd-bede-58f049f5665b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:52.412405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f0089-162b-4edd-bede-58f049f5665b admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:52.413065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f0089-162b-4edd-bede-58f049f5665b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:52.415090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f0089-162b-4edd-bede-58f049f5665b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:52.424083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:52.531767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1291/2579] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:58:52 2022] POST /identity/v3/users => generated 302 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:58:52.537538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce40f5-2b2f-4344-b502-9388842453a9 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:52.549464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce40f5-2b2f-4344-b502-9388842453a9 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:52.554769 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd4e7f-b42a-4aec-b890-bddcae5cc5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:52.557878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd4e7f-b42a-4aec-b890-bddcae5cc5bc admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:52.557878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd4e7f-b42a-4aec-b890-bddcae5cc5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:52.557878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd4e7f-b42a-4aec-b890-bddcae5cc5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:52.557878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd4e7f-b42a-4aec-b890-bddcae5cc5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:52.558848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd4e7f-b42a-4aec-b890-bddcae5cc5bc admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:52.572533 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1289/2580] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:58:52.580393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f0089-162b-4edd-bede-58f049f5665b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:52.588191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f0089-162b-4edd-bede-58f049f5665b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:52.593636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8dcee-ff8c-400d-b20a-0c6521fdf2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:52.594271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8dcee-ff8c-400d-b20a-0c6521fdf2df admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:52.594582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8dcee-ff8c-400d-b20a-0c6521fdf2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:52.594861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8dcee-ff8c-400d-b20a-0c6521fdf2df admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:52.595543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8dcee-ff8c-400d-b20a-0c6521fdf2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:52.597430 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8dcee-ff8c-400d-b20a-0c6521fdf2df admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:52.621672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1292/2581] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:58:52.622983 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd4e7f-b42a-4aec-b890-bddcae5cc5bc admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:52.633483 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd4e7f-b42a-4aec-b890-bddcae5cc5bc admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:52.639392 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa79c1-67fc-418a-a838-4f668da6d3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:52.640205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa79c1-67fc-418a-a838-4f668da6d3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:52.640492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa79c1-67fc-418a-a838-4f668da6d3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:52.640771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa79c1-67fc-418a-a838-4f668da6d3ea admin admin] PATH_INFO: `/v3/projects/621e0b99ebfc415288e2d856b5c6bf64/users/72bc3579dc514f16bb267a6d66a17b19/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:52.659018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa79c1-67fc-418a-a838-4f668da6d3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=621e0b99ebfc415288e2d856b5c6bf64, user_id=72bc3579dc514f16bb267a6d66a17b19, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:52.661843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa79c1-67fc-418a-a838-4f668da6d3ea admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:52.799435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1290/2582] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:58:52 2022] PUT /identity/v3/projects/621e0b99ebfc415288e2d856b5c6bf64/users/72bc3579dc514f16bb267a6d66a17b19/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:58:52.804243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3025de-0655-497d-a219-efae43d12c93 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:52.804526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3025de-0655-497d-a219-efae43d12c93 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:52.804829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3025de-0655-497d-a219-efae43d12c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:52.825515 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-da3025de-0655-497d-a219-efae43d12c93 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:52.827391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-da3025de-0655-497d-a219-efae43d12c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:52.888384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1293/2583] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:52.898813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa79c1-67fc-418a-a838-4f668da6d3ea admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:52.900633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3bfa79c1-67fc-418a-a838-4f668da6d3ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:52.966364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa79c1-67fc-418a-a838-4f668da6d3ea admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:52.970143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da9990-c07f-4f10-866a-e7b271f3bba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:52.970788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da9990-c07f-4f10-866a-e7b271f3bba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:52.971496 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da9990-c07f-4f10-866a-e7b271f3bba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:52.971773 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da9990-c07f-4f10-866a-e7b271f3bba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:52.981698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da9990-c07f-4f10-866a-e7b271f3bba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:52.983229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da9990-c07f-4f10-866a-e7b271f3bba9 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:52.993798 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1291/2584] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:58:54.948761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef01e2b-d328-470a-a7a0-8d3d1a3ee39f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:54.949322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef01e2b-d328-470a-a7a0-8d3d1a3ee39f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:54.949322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef01e2b-d328-470a-a7a0-8d3d1a3ee39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:54.986201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7ef01e2b-d328-470a-a7a0-8d3d1a3ee39f None None] MFA Rules not processed for user `72bc3579dc514f16bb267a6d66a17b19`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:54.988826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7ef01e2b-d328-470a-a7a0-8d3d1a3ee39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:55.085711 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1294/2585] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:58:54 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:55.096246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da9990-c07f-4f10-866a-e7b271f3bba9 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:55.107397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da9990-c07f-4f10-866a-e7b271f3bba9 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:55.112918 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34c223-b941-4a0d-bc6f-10959c07345f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:55.113527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34c223-b941-4a0d-bc6f-10959c07345f admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:55.113808 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34c223-b941-4a0d-bc6f-10959c07345f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:55.114078 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34c223-b941-4a0d-bc6f-10959c07345f admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:55.114770 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34c223-b941-4a0d-bc6f-10959c07345f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:55.116683 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34c223-b941-4a0d-bc6f-10959c07345f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:55.231485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1292/2586] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:58:55 2022] POST /identity/v3/projects => generated 405 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:58:55.236155 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef01e2b-d328-470a-a7a0-8d3d1a3ee39f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:55.247928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef01e2b-d328-470a-a7a0-8d3d1a3ee39f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:55.309828 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bfd97-446b-431e-8203-b55ffa803ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:55.310445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bfd97-446b-431e-8203-b55ffa803ced admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:55.310738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bfd97-446b-431e-8203-b55ffa803ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:55.311008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bfd97-446b-431e-8203-b55ffa803ced admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:55.311656 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bfd97-446b-431e-8203-b55ffa803ced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:55.314059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bfd97-446b-431e-8203-b55ffa803ced admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:55.400862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:55.406740 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1295/2587] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:58:55 2022] POST /identity/v3/users => generated 309 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:58:55.411010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34c223-b941-4a0d-bc6f-10959c07345f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:55.422450 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34c223-b941-4a0d-bc6f-10959c07345f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:55.426994 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6dc7f-1502-4400-abcd-a5c852f5ae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:55.427614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6dc7f-1502-4400-abcd-a5c852f5ae5e admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:55.427890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6dc7f-1502-4400-abcd-a5c852f5ae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:55.428158 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6dc7f-1502-4400-abcd-a5c852f5ae5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:55.428960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6dc7f-1502-4400-abcd-a5c852f5ae5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:55.430559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6dc7f-1502-4400-abcd-a5c852f5ae5e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:55.441990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1293/2588] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:58:55.446282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bfd97-446b-431e-8203-b55ffa803ced admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:55.458035 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bfd97-446b-431e-8203-b55ffa803ced admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:55.467045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb8d0b-0aa7-490b-8082-6a5d6e8e5db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:55.467045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8d0b-0aa7-490b-8082-6a5d6e8e5db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:55.467045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8d0b-0aa7-490b-8082-6a5d6e8e5db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:55.467045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8d0b-0aa7-490b-8082-6a5d6e8e5db7 admin admin] PATH_INFO: `/v3/projects/ad21007074b44c72ad3879592d63e9fb/users/6cf32096589541898114b94352b09fff/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:55.482006 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb8d0b-0aa7-490b-8082-6a5d6e8e5db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad21007074b44c72ad3879592d63e9fb, user_id=6cf32096589541898114b94352b09fff, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:55.484728 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb8d0b-0aa7-490b-8082-6a5d6e8e5db7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:55.585834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1296/2589] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:58:55 2022] PUT /identity/v3/projects/ad21007074b44c72ad3879592d63e9fb/users/6cf32096589541898114b94352b09fff/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:58:55.590620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6dc7f-1502-4400-abcd-a5c852f5ae5e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:55.602615 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6dc7f-1502-4400-abcd-a5c852f5ae5e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:55.664603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb8d0b-0aa7-490b-8082-6a5d6e8e5db7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:55.668571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5dbb8d0b-0aa7-490b-8082-6a5d6e8e5db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:55.674258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990c420-bcc7-459e-b5d5-fa7b5c3cca36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:55.674964 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990c420-bcc7-459e-b5d5-fa7b5c3cca36 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:55.675386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990c420-bcc7-459e-b5d5-fa7b5c3cca36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:55.675670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990c420-bcc7-459e-b5d5-fa7b5c3cca36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:55.676533 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990c420-bcc7-459e-b5d5-fa7b5c3cca36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:55.678187 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990c420-bcc7-459e-b5d5-fa7b5c3cca36 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:55.684151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb8d0b-0aa7-490b-8082-6a5d6e8e5db7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:55.688918 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d792f-a843-49a7-83b7-5e2a47f0262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:55.689557 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d792f-a843-49a7-83b7-5e2a47f0262c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:55.689836 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d792f-a843-49a7-83b7-5e2a47f0262c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:55.690109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d792f-a843-49a7-83b7-5e2a47f0262c admin admin] PATH_INFO: `/v3/users/d80c63ef5d3848749ae96c57c36488e0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:55.691083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1294/2590] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:58:55.691300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d792f-a843-49a7-83b7-5e2a47f0262c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d80c63ef5d3848749ae96c57c36488e0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:55.693364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d792f-a843-49a7-83b7-5e2a47f0262c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:55.695517 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990c420-bcc7-459e-b5d5-fa7b5c3cca36 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:55.707662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990c420-bcc7-459e-b5d5-fa7b5c3cca36 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:55.712699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb29c5d-56bb-4ac0-aa41-52fdade797b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:55.713934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb29c5d-56bb-4ac0-aa41-52fdade797b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:55.714626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb29c5d-56bb-4ac0-aa41-52fdade797b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:55.714912 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb29c5d-56bb-4ac0-aa41-52fdade797b8 admin admin] PATH_INFO: `/v3/projects/ad21007074b44c72ad3879592d63e9fb/users/6cf32096589541898114b94352b09fff/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:55.717104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb29c5d-56bb-4ac0-aa41-52fdade797b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad21007074b44c72ad3879592d63e9fb, user_id=6cf32096589541898114b94352b09fff, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:55.719790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb29c5d-56bb-4ac0-aa41-52fdade797b8 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:55.727760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:55.735347 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1295/2591] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:58:55 2022] PUT /identity/v3/projects/ad21007074b44c72ad3879592d63e9fb/users/6cf32096589541898114b94352b09fff/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:58:55.983922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb29c5d-56bb-4ac0-aa41-52fdade797b8 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:55.986637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5bb29c5d-56bb-4ac0-aa41-52fdade797b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:56.039950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9c5d792f-a843-49a7-83b7-5e2a47f0262c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd80c63ef5d3848749ae96c57c36488e0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:56.072511 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9c5d792f-a843-49a7-83b7-5e2a47f0262c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd80c63ef5d3848749ae96c57c36488e0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:56.086143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9c5d792f-a843-49a7-83b7-5e2a47f0262c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd80c63ef5d3848749ae96c57c36488e0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:56.093272 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb29c5d-56bb-4ac0-aa41-52fdade797b8 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:56.097640 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462e7ec-cdb7-4138-8c88-b1cef9e40b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:56.098322 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462e7ec-cdb7-4138-8c88-b1cef9e40b40 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:56.098563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462e7ec-cdb7-4138-8c88-b1cef9e40b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:56.100906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462e7ec-cdb7-4138-8c88-b1cef9e40b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:56.100906 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d462e7ec-cdb7-4138-8c88-b1cef9e40b40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:56.182822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9c5d792f-a843-49a7-83b7-5e2a47f0262c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd80c63ef5d3848749ae96c57c36488e0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:56.183926 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9c5d792f-a843-49a7-83b7-5e2a47f0262c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd80c63ef5d3848749ae96c57c36488e0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:56.189748 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462e7ec-cdb7-4138-8c88-b1cef9e40b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:56.192127 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462e7ec-cdb7-4138-8c88-b1cef9e40b40 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:56.203517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1297/2592] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:58:55 2022] DELETE /identity/v3/users/d80c63ef5d3848749ae96c57c36488e0 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:58:56.205928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efa63a-6549-4be0-9505-4647216ec392 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:56.206292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efa63a-6549-4be0-9505-4647216ec392 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:56.206682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efa63a-6549-4be0-9505-4647216ec392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:56.207473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1296/2593] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:58:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:58:56.229560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c4efa63a-6549-4be0-9505-4647216ec392 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:56.231199 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c4efa63a-6549-4be0-9505-4647216ec392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:56.322271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1298/2594] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:56.337289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462e7ec-cdb7-4138-8c88-b1cef9e40b40 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:56.354767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462e7ec-cdb7-4138-8c88-b1cef9e40b40 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:56.414278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129c115-76fd-4b4c-8f52-2f610806f7cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:56.414999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129c115-76fd-4b4c-8f52-2f610806f7cd service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:56.415283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129c115-76fd-4b4c-8f52-2f610806f7cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:56.415679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129c115-76fd-4b4c-8f52-2f610806f7cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:56.435468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129c115-76fd-4b4c-8f52-2f610806f7cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:56.438492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129c115-76fd-4b4c-8f52-2f610806f7cd service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:56.454604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1297/2595] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:58:57.326016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efa63a-6549-4be0-9505-4647216ec392 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:57.328938 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c4efa63a-6549-4be0-9505-4647216ec392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:57.346015 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efa63a-6549-4be0-9505-4647216ec392 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:57.352031 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:57.353316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:57.353527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:57.353691 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] PATH_INFO: `/v3/projects/71a4e04b99e547de83c1e2359f5ce1fa` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:57.355872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a4e04b99e547de83c1e2359f5ce1fa)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:57.357917 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:57.505273 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:57.513346 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a4e04b99e547de83c1e2359f5ce1fa', 'request_id': 'req-48447397-e060-4d2c-aa18-c3fa8e71da2b'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:57.520372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a4e04b99e547de83c1e2359f5ce1fa', 'request_id': 'req-48447397-e060-4d2c-aa18-c3fa8e71da2b'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:57.551758 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d078fd-de3c-4343-8489-9b2c8d14d594 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:57.552309 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d078fd-de3c-4343-8489-9b2c8d14d594 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:57.552532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d078fd-de3c-4343-8489-9b2c8d14d594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:57.585989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-24d078fd-de3c-4343-8489-9b2c8d14d594 None None] MFA Rules not processed for user `6cf32096589541898114b94352b09fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:57.587525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-24d078fd-de3c-4343-8489-9b2c8d14d594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:57.634703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a4e04b99e547de83c1e2359f5ce1fa', 'request_id': 'req-48447397-e060-4d2c-aa18-c3fa8e71da2b'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:57.654366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] The token cache is being invalidate because project 71a4e04b99e547de83c1e2359f5ce1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:58:57.655275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:57.656660 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:57.659173 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:57.661358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1299/2596] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:58:57 2022] DELETE /identity/v3/projects/71a4e04b99e547de83c1e2359f5ce1fa => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:58:57.671510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:57.717936 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:57.718108 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1298/2597] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:58:57 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:57.779462 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48447397-e060-4d2c-aa18-c3fa8e71da2b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:57.785215 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134729ee-e048-44ac-ab82-dd4fe9f54e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:57.786368 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134729ee-e048-44ac-ab82-dd4fe9f54e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:57.786857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134729ee-e048-44ac-ab82-dd4fe9f54e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:57.787376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134729ee-e048-44ac-ab82-dd4fe9f54e4d admin admin] PATH_INFO: `/v3/users/ec241162ccf0462488c962fa4dc8375d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:57.792570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134729ee-e048-44ac-ab82-dd4fe9f54e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec241162ccf0462488c962fa4dc8375d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:57.797228 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134729ee-e048-44ac-ab82-dd4fe9f54e4d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:58.109322 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-134729ee-e048-44ac-ab82-dd4fe9f54e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec241162ccf0462488c962fa4dc8375d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:58.124413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-134729ee-e048-44ac-ab82-dd4fe9f54e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec241162ccf0462488c962fa4dc8375d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:58.131672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-134729ee-e048-44ac-ab82-dd4fe9f54e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec241162ccf0462488c962fa4dc8375d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:58.195692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-134729ee-e048-44ac-ab82-dd4fe9f54e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec241162ccf0462488c962fa4dc8375d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:58.196509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-134729ee-e048-44ac-ab82-dd4fe9f54e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec241162ccf0462488c962fa4dc8375d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:58.205702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1300/2598] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:58:57 2022] DELETE /identity/v3/users/ec241162ccf0462488c962fa4dc8375d => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:58:58.818048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d078fd-de3c-4343-8489-9b2c8d14d594 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:58.848326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d078fd-de3c-4343-8489-9b2c8d14d594 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:58.939980 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:58.940725 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:58.941686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:58.942044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] PATH_INFO: `/v3/projects/0f0421576b8c4c71be67ff4bacbfb35c` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:58.943936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0421576b8c4c71be67ff4bacbfb35c)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:58.946840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:59.103561 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0421576b8c4c71be67ff4bacbfb35c', 'request_id': 'req-e639ac77-0c28-41bc-8a08-aa0a344e097f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:59.185516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0421576b8c4c71be67ff4bacbfb35c', 'request_id': 'req-e639ac77-0c28-41bc-8a08-aa0a344e097f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:59.193812 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0421576b8c4c71be67ff4bacbfb35c', 'request_id': 'req-e639ac77-0c28-41bc-8a08-aa0a344e097f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:59.213992 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] The token cache is being invalidate because project 0f0421576b8c4c71be67ff4bacbfb35c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:58:59.214349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:59.215125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e639ac77-0c28-41bc-8a08-aa0a344e097f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:58:59.217396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1299/2599] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:58:58 2022] DELETE /identity/v3/projects/0f0421576b8c4c71be67ff4bacbfb35c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:58:59.224578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a56cd-7278-42eb-ae96-44b66d5f4b18 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:59.225014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a56cd-7278-42eb-ae96-44b66d5f4b18 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:59.225227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a56cd-7278-42eb-ae96-44b66d5f4b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:59.257398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d28a56cd-7278-42eb-ae96-44b66d5f4b18 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:59.258704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d28a56cd-7278-42eb-ae96-44b66d5f4b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:59.327558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:58:59.352034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1301/2600] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:58:59.364119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1ca8c-df3d-47d9-bdd8-7621bc55d14c None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:59.364706 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1ca8c-df3d-47d9-bdd8-7621bc55d14c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:59.366049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1ca8c-df3d-47d9-bdd8-7621bc55d14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:59.396444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-bfb1ca8c-df3d-47d9-bdd8-7621bc55d14c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:58:59.398823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bfb1ca8c-df3d-47d9-bdd8-7621bc55d14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:59.416374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a56cd-7278-42eb-ae96-44b66d5f4b18 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:59.423354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1300/2601] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:58:59.430560 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a56cd-7278-42eb-ae96-44b66d5f4b18 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:59.443791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fb5d7-3a7f-41d9-a56b-5cd43c2e7fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:59.446742 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb5d7-3a7f-41d9-a56b-5cd43c2e7fca admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:59.447071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb5d7-3a7f-41d9-a56b-5cd43c2e7fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:59.447351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb5d7-3a7f-41d9-a56b-5cd43c2e7fca admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:59.448556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fb5d7-3a7f-41d9-a56b-5cd43c2e7fca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:58:59.448862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fb5d7-3a7f-41d9-a56b-5cd43c2e7fca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:59.450665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fb5d7-3a7f-41d9-a56b-5cd43c2e7fca admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:59.470628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1302/2602] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:58:59.476780 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1ca8c-df3d-47d9-bdd8-7621bc55d14c None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:59.487681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1ca8c-df3d-47d9-bdd8-7621bc55d14c None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:59.497551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf6810-5c59-4ac9-97b1-52fb4dd5e16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:59.498357 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf6810-5c59-4ac9-97b1-52fb4dd5e16b admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:59.498678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf6810-5c59-4ac9-97b1-52fb4dd5e16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:59.498946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf6810-5c59-4ac9-97b1-52fb4dd5e16b admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:59.499613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf6810-5c59-4ac9-97b1-52fb4dd5e16b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:59.501531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf6810-5c59-4ac9-97b1-52fb4dd5e16b admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:59.637557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1301/2603] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:58:59 2022] POST /identity/v3/projects => generated 357 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:58:59.644235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fb5d7-3a7f-41d9-a56b-5cd43c2e7fca admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:59.669783 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fb5d7-3a7f-41d9-a56b-5cd43c2e7fca admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:59.799726 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f9b66-663a-40ee-9484-70a34ed134d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:59.800406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9b66-663a-40ee-9484-70a34ed134d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:59.800687 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9b66-663a-40ee-9484-70a34ed134d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:59.800957 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9b66-663a-40ee-9484-70a34ed134d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:59.801613 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f9b66-663a-40ee-9484-70a34ed134d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:59.803679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f9b66-663a-40ee-9484-70a34ed134d4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:59.830282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf6810-5c59-4ac9-97b1-52fb4dd5e16b admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:59.836724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-eddf6810-5c59-4ac9-97b1-52fb4dd5e16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:58:59.862191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf6810-5c59-4ac9-97b1-52fb4dd5e16b admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:59.875334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:59.876016 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:59.876301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:59.876559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] PATH_INFO: `/v3/users/72bc3579dc514f16bb267a6d66a17b19` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:59.879282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72bc3579dc514f16bb267a6d66a17b19)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:59.884095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:58:59.904385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1303/2604] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:58:59 2022] POST /identity/v3/users => generated 279 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:58:59.919237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f9b66-663a-40ee-9484-70a34ed134d4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:58:59.952425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f9b66-663a-40ee-9484-70a34ed134d4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:58:59.960850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125a8df-b9cb-4628-b327-cf5406739934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:58:59.967398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125a8df-b9cb-4628-b327-cf5406739934 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:58:59.967398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125a8df-b9cb-4628-b327-cf5406739934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:58:59.967398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125a8df-b9cb-4628-b327-cf5406739934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:58:59.967398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125a8df-b9cb-4628-b327-cf5406739934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:58:59.967398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125a8df-b9cb-4628-b327-cf5406739934 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:00.001796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1304/2605] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:00.005825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125a8df-b9cb-4628-b327-cf5406739934 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:00.026793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125a8df-b9cb-4628-b327-cf5406739934 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:00.040198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd1223-7b85-42bb-9a44-c7856c58d16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:00.043128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1223-7b85-42bb-9a44-c7856c58d16e admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:00.043128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1223-7b85-42bb-9a44-c7856c58d16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:00.043128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1223-7b85-42bb-9a44-c7856c58d16e admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:00.047842 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd1223-7b85-42bb-9a44-c7856c58d16e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:00.050810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd1223-7b85-42bb-9a44-c7856c58d16e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:00.075443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1305/2606] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:00.077692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd1223-7b85-42bb-9a44-c7856c58d16e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:00.099039 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd1223-7b85-42bb-9a44-c7856c58d16e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:00.109326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d53458-0c63-4b5a-8ef5-89c9649d332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:00.112607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d53458-0c63-4b5a-8ef5-89c9649d332f admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:00.112607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d53458-0c63-4b5a-8ef5-89c9649d332f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:00.112607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d53458-0c63-4b5a-8ef5-89c9649d332f admin admin] PATH_INFO: `/v3/projects/0c36ba46e6c54fadb6e4acd8c0c11656/users/d03daee53c5444fca3aebee7c2f60d38/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:00.142216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d53458-0c63-4b5a-8ef5-89c9649d332f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c36ba46e6c54fadb6e4acd8c0c11656, user_id=d03daee53c5444fca3aebee7c2f60d38, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:00.147413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d53458-0c63-4b5a-8ef5-89c9649d332f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:00.173419 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:00.184522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72bc3579dc514f16bb267a6d66a17b19'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:00.199190 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72bc3579dc514f16bb267a6d66a17b19'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:00.209985 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72bc3579dc514f16bb267a6d66a17b19'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:00.299637 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1306/2607] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:00 2022] PUT /identity/v3/projects/0c36ba46e6c54fadb6e4acd8c0c11656/users/d03daee53c5444fca3aebee7c2f60d38/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:00.311411 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4fadc-b5a5-4e16-a937-8b229e23733d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:00.311722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4fadc-b5a5-4e16-a937-8b229e23733d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:00.312002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4fadc-b5a5-4e16-a937-8b229e23733d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:00.327425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72bc3579dc514f16bb267a6d66a17b19'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:00.331316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72bc3579dc514f16bb267a6d66a17b19'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:00.360670 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1302/2608] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:58:59 2022] DELETE /identity/v3/users/72bc3579dc514f16bb267a6d66a17b19 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:00.380246 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-74d4fadc-b5a5-4e16-a937-8b229e23733d None None] MFA Rules not processed for user `d03daee53c5444fca3aebee7c2f60d38`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:00.382524 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-74d4fadc-b5a5-4e16-a937-8b229e23733d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:00.496581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1307/2609] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:59:00.501475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:00.503070 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:00.618212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ade14-62bd-436f-8dd1-531ef1403fd5 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:00.623867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ccb44-3dcd-48e8-9dba-3182d5a076e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:00.624523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ccb44-3dcd-48e8-9dba-3182d5a076e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:00.624845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ccb44-3dcd-48e8-9dba-3182d5a076e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:00.625123 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ccb44-3dcd-48e8-9dba-3182d5a076e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:00.625783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ccb44-3dcd-48e8-9dba-3182d5a076e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:00.627769 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ccb44-3dcd-48e8-9dba-3182d5a076e7 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:00.735679 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4fadc-b5a5-4e16-a937-8b229e23733d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:00.737225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-74d4fadc-b5a5-4e16-a937-8b229e23733d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:00.756468 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4fadc-b5a5-4e16-a937-8b229e23733d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:00.765366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1303/2610] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:00 2022] POST /identity/v3/projects => generated 357 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:59:00.769963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ccb44-3dcd-48e8-9dba-3182d5a076e7 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:00.783129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:00.783702 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:00.784144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:00.784418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] PATH_INFO: `/v3/projects/621e0b99ebfc415288e2d856b5c6bf64` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:00.785648 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=621e0b99ebfc415288e2d856b5c6bf64)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:00.787736 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:00.790791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:00.853063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ccb44-3dcd-48e8-9dba-3182d5a076e7 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:00.858239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31beb3c5-b39e-4707-849b-501c4056978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:00.859066 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beb3c5-b39e-4707-849b-501c4056978a admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:00.859482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beb3c5-b39e-4707-849b-501c4056978a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:00.859955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beb3c5-b39e-4707-849b-501c4056978a admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:00.860386 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31beb3c5-b39e-4707-849b-501c4056978a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:00.862342 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31beb3c5-b39e-4707-849b-501c4056978a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:00.927909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '621e0b99ebfc415288e2d856b5c6bf64', 'request_id': 'req-23b17817-56d5-49ff-bfd3-194ffdf497cd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:00.935665 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '621e0b99ebfc415288e2d856b5c6bf64', 'request_id': 'req-23b17817-56d5-49ff-bfd3-194ffdf497cd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:00.979738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1304/2611] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:00 2022] POST /identity/v3/users => generated 285 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:00.984040 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31beb3c5-b39e-4707-849b-501c4056978a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:00.996802 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31beb3c5-b39e-4707-849b-501c4056978a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:01.006961 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '621e0b99ebfc415288e2d856b5c6bf64', 'request_id': 'req-23b17817-56d5-49ff-bfd3-194ffdf497cd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:01.025849 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:01.026162 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] The token cache is being invalidate because project 621e0b99ebfc415288e2d856b5c6bf64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:01.026490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:01.027240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:01.028796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1308/2612] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:00 2022] DELETE /identity/v3/projects/621e0b99ebfc415288e2d856b5c6bf64 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:01.032668 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:01.034186 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:01.075665 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499eee5-afdc-4072-8e45-7031a87f181b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:01.076286 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499eee5-afdc-4072-8e45-7031a87f181b admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:01.076565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499eee5-afdc-4072-8e45-7031a87f181b admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:01.076836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499eee5-afdc-4072-8e45-7031a87f181b admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:01.077582 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499eee5-afdc-4072-8e45-7031a87f181b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:01.079258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499eee5-afdc-4072-8e45-7031a87f181b admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:01.103090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1305/2613] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:59:01.109025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499eee5-afdc-4072-8e45-7031a87f181b admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:01.111049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7499eee5-afdc-4072-8e45-7031a87f181b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:01.125253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499eee5-afdc-4072-8e45-7031a87f181b admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:01.127427 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b17817-56d5-49ff-bfd3-194ffdf497cd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:01.130351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7e61d-1041-4db9-82b5-26d28e9b60db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:01.131083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e61d-1041-4db9-82b5-26d28e9b60db admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:01.137030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e61d-1041-4db9-82b5-26d28e9b60db admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:01.137030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e61d-1041-4db9-82b5-26d28e9b60db admin admin] PATH_INFO: `/v3/projects/40dc374c0d974579a11ab2da7d492a58/users/8ba82e28a63c4012a08da9e882947d33/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:01.137358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:01.137358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:01.137358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:01.137358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] PATH_INFO: `/v3/users/6cf32096589541898114b94352b09fff` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:01.137887 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cf32096589541898114b94352b09fff)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:01.139016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:01.150253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7e61d-1041-4db9-82b5-26d28e9b60db admin admin] RBAC: Authorizing `identity:create_grant(project_id=40dc374c0d974579a11ab2da7d492a58, user_id=8ba82e28a63c4012a08da9e882947d33, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:01.153634 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7e61d-1041-4db9-82b5-26d28e9b60db admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:01.323594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1306/2614] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:01 2022] PUT /identity/v3/projects/40dc374c0d974579a11ab2da7d492a58/users/8ba82e28a63c4012a08da9e882947d33/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:01.328568 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7e61d-1041-4db9-82b5-26d28e9b60db admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:01.413290 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7e61d-1041-4db9-82b5-26d28e9b60db admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:01.419575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f276d2-dc93-401c-9181-e5b93fd71607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:01.420203 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f276d2-dc93-401c-9181-e5b93fd71607 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:01.420484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f276d2-dc93-401c-9181-e5b93fd71607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:01.420759 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f276d2-dc93-401c-9181-e5b93fd71607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:01.421421 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f276d2-dc93-401c-9181-e5b93fd71607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:01.423119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f276d2-dc93-401c-9181-e5b93fd71607 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:01.435393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1307/2615] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:59:01.439958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f276d2-dc93-401c-9181-e5b93fd71607 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:01.451206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f276d2-dc93-401c-9181-e5b93fd71607 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:01.456754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eac7b1-2504-4e04-ac1b-5e4208d30c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:01.457455 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac7b1-2504-4e04-ac1b-5e4208d30c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:01.457735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac7b1-2504-4e04-ac1b-5e4208d30c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:01.458031 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac7b1-2504-4e04-ac1b-5e4208d30c61 admin admin] PATH_INFO: `/v3/projects/40dc374c0d974579a11ab2da7d492a58/users/8ba82e28a63c4012a08da9e882947d33/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:01.460226 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eac7b1-2504-4e04-ac1b-5e4208d30c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40dc374c0d974579a11ab2da7d492a58, user_id=8ba82e28a63c4012a08da9e882947d33, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:01.463205 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eac7b1-2504-4e04-ac1b-5e4208d30c61 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:01.476045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1308/2616] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:01 2022] PUT /identity/v3/projects/40dc374c0d974579a11ab2da7d492a58/users/8ba82e28a63c4012a08da9e882947d33/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:01.486679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bafd9f-ec6e-4979-b6fc-5cf87aa935b5 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:01.486963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bafd9f-ec6e-4979-b6fc-5cf87aa935b5 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:01.487238 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bafd9f-ec6e-4979-b6fc-5cf87aa935b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:01.522118 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-34bafd9f-ec6e-4979-b6fc-5cf87aa935b5 None None] MFA Rules not processed for user `8ba82e28a63c4012a08da9e882947d33`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:01.531925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-34bafd9f-ec6e-4979-b6fc-5cf87aa935b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:01.608898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1309/2617] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:59:01.620500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bafd9f-ec6e-4979-b6fc-5cf87aa935b5 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:01.622198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-34bafd9f-ec6e-4979-b6fc-5cf87aa935b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:01.627367 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cf32096589541898114b94352b09fff'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:01.631986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:01.652526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cf32096589541898114b94352b09fff'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:01.665159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cf32096589541898114b94352b09fff'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:01.712358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bafd9f-ec6e-4979-b6fc-5cf87aa935b5 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:01.716641 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa1130-7f41-4d8e-b72a-a5cdffef76b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:01.717249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1130-7f41-4d8e-b72a-a5cdffef76b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:01.717627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1130-7f41-4d8e-b72a-a5cdffef76b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:01.717863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1130-7f41-4d8e-b72a-a5cdffef76b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:01.755621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa1130-7f41-4d8e-b72a-a5cdffef76b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:01.757278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa1130-7f41-4d8e-b72a-a5cdffef76b2 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:01.768826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1310/2618] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:59:01.789960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cf32096589541898114b94352b09fff'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:01.790703 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cf32096589541898114b94352b09fff'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:01.798569 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1309/2619] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:59:01 2022] DELETE /identity/v3/users/6cf32096589541898114b94352b09fff => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:02.004676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa1130-7f41-4d8e-b72a-a5cdffef76b2 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:02.023651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa1130-7f41-4d8e-b72a-a5cdffef76b2 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:02.077590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:02.079789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:02.096765 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc21c5-8a4e-48de-81be-1dabe352abfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:02.097397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc21c5-8a4e-48de-81be-1dabe352abfc service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:02.097674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc21c5-8a4e-48de-81be-1dabe352abfc service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:02.097946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc21c5-8a4e-48de-81be-1dabe352abfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:02.100616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a6bc21c5-8a4e-48de-81be-1dabe352abfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:02.101381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:02.173907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc21c5-8a4e-48de-81be-1dabe352abfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:02.175527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc21c5-8a4e-48de-81be-1dabe352abfc service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:02.186962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1311/2620] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:02.239160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d874088-8d38-409b-ac26-1b69296f5c04 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:02.244198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:02.245420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:02.245694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:02.245960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] PATH_INFO: `/v3/projects/ad21007074b44c72ad3879592d63e9fb` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:02.247156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad21007074b44c72ad3879592d63e9fb)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:02.249082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:02.391938 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad21007074b44c72ad3879592d63e9fb', 'request_id': 'req-1487138c-66a6-48bf-9016-dea8359d38d7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:02.398955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad21007074b44c72ad3879592d63e9fb', 'request_id': 'req-1487138c-66a6-48bf-9016-dea8359d38d7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:02.470168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad21007074b44c72ad3879592d63e9fb', 'request_id': 'req-1487138c-66a6-48bf-9016-dea8359d38d7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:02.485812 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] The token cache is being invalidate because project ad21007074b44c72ad3879592d63e9fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:02.486157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:02.486817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1487138c-66a6-48bf-9016-dea8359d38d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:02.488366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1310/2621] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:02 2022] DELETE /identity/v3/projects/ad21007074b44c72ad3879592d63e9fb => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:02.495045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d481c-b86c-4f13-b9b4-b9466b29a315 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:02.495528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d481c-b86c-4f13-b9b4-b9466b29a315 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:02.512741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d481c-b86c-4f13-b9b4-b9466b29a315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:02.521926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:02.577039 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-679d481c-b86c-4f13-b9b4-b9466b29a315 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:02.581437 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-679d481c-b86c-4f13-b9b4-b9466b29a315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:02.664068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1312/2622] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:02.672940 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426355c-6460-411c-8dcf-c7f8367ad65b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:02.674046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426355c-6460-411c-8dcf-c7f8367ad65b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:02.674046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426355c-6460-411c-8dcf-c7f8367ad65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:02.698735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d426355c-6460-411c-8dcf-c7f8367ad65b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:02.700159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d426355c-6460-411c-8dcf-c7f8367ad65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:02.708691 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1311/2623] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:02.714904 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d481c-b86c-4f13-b9b4-b9466b29a315 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:02.733026 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d481c-b86c-4f13-b9b4-b9466b29a315 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:02.743581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeff3f6-3f58-4916-872b-b05dfc9229df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:02.744209 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeff3f6-3f58-4916-872b-b05dfc9229df admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:02.744518 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeff3f6-3f58-4916-872b-b05dfc9229df admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:02.744791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeff3f6-3f58-4916-872b-b05dfc9229df admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:02.745459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeff3f6-3f58-4916-872b-b05dfc9229df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:59:02.745928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeff3f6-3f58-4916-872b-b05dfc9229df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:02.748538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeff3f6-3f58-4916-872b-b05dfc9229df admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:02.767010 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1313/2624] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:59:02.773857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426355c-6460-411c-8dcf-c7f8367ad65b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:02.788305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426355c-6460-411c-8dcf-c7f8367ad65b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:02.795757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271125d-f01a-4821-84b5-5d1d9f80a869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:02.796384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271125d-f01a-4821-84b5-5d1d9f80a869 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:02.796663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271125d-f01a-4821-84b5-5d1d9f80a869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:02.796935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271125d-f01a-4821-84b5-5d1d9f80a869 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:02.797628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271125d-f01a-4821-84b5-5d1d9f80a869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:02.799573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271125d-f01a-4821-84b5-5d1d9f80a869 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:02.906530 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1312/2625] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:02 2022] POST /identity/v3/projects => generated 363 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:02.910998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeff3f6-3f58-4916-872b-b05dfc9229df admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:02.921555 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeff3f6-3f58-4916-872b-b05dfc9229df admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:02.983377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe6597-77ee-488a-b087-e6e02cad980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:02.984029 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe6597-77ee-488a-b087-e6e02cad980a admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:02.984303 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe6597-77ee-488a-b087-e6e02cad980a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:02.984572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe6597-77ee-488a-b087-e6e02cad980a admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:02.985253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe6597-77ee-488a-b087-e6e02cad980a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:02.987288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe6597-77ee-488a-b087-e6e02cad980a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:03.151876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1314/2626] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:02 2022] POST /identity/v3/users => generated 282 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:59:03.156326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271125d-f01a-4821-84b5-5d1d9f80a869 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:03.167676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271125d-f01a-4821-84b5-5d1d9f80a869 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:03.172644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613d6eb-e90f-474a-8efe-36c50bda8f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:03.173195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613d6eb-e90f-474a-8efe-36c50bda8f47 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:03.173471 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613d6eb-e90f-474a-8efe-36c50bda8f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:03.173748 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613d6eb-e90f-474a-8efe-36c50bda8f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:03.174409 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613d6eb-e90f-474a-8efe-36c50bda8f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:03.176258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613d6eb-e90f-474a-8efe-36c50bda8f47 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:03.188236 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1313/2627] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:03.193269 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe6597-77ee-488a-b087-e6e02cad980a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:03.205013 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe6597-77ee-488a-b087-e6e02cad980a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:03.211030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d98ca-a6ee-4359-9eb8-8d8f7793d604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:03.211654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d98ca-a6ee-4359-9eb8-8d8f7793d604 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:03.211936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d98ca-a6ee-4359-9eb8-8d8f7793d604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:03.212212 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d98ca-a6ee-4359-9eb8-8d8f7793d604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:03.212873 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d98ca-a6ee-4359-9eb8-8d8f7793d604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:03.215334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d98ca-a6ee-4359-9eb8-8d8f7793d604 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:03.227627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1315/2628] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:03.238686 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613d6eb-e90f-474a-8efe-36c50bda8f47 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:03.249662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613d6eb-e90f-474a-8efe-36c50bda8f47 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:03.254544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c878e-0205-4978-8438-e7dab81ffd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:03.255245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c878e-0205-4978-8438-e7dab81ffd69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:03.255550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c878e-0205-4978-8438-e7dab81ffd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:03.255829 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c878e-0205-4978-8438-e7dab81ffd69 admin admin] PATH_INFO: `/v3/projects/bbf8221aac334b109f409c0e68b73b67/users/51adca6e07034efaaa8c5c35f6c3dc85/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:03.259514 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:03.272296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c878e-0205-4978-8438-e7dab81ffd69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf8221aac334b109f409c0e68b73b67, user_id=51adca6e07034efaaa8c5c35f6c3dc85, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:03.275117 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c878e-0205-4978-8438-e7dab81ffd69 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:03.357269 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1314/2629] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:03 2022] PUT /identity/v3/projects/bbf8221aac334b109f409c0e68b73b67/users/51adca6e07034efaaa8c5c35f6c3dc85/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:03.368938 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ffce2-9734-4e53-9a8f-d1e03c1dc411 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:03.369233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ffce2-9734-4e53-9a8f-d1e03c1dc411 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:03.369510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ffce2-9734-4e53-9a8f-d1e03c1dc411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:03.400288 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f85ffce2-9734-4e53-9a8f-d1e03c1dc411 None None] MFA Rules not processed for user `51adca6e07034efaaa8c5c35f6c3dc85`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:03.402448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f85ffce2-9734-4e53-9a8f-d1e03c1dc411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:03.447330 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:03.477180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1316/2630] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:03.482466 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c878e-0205-4978-8438-e7dab81ffd69 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:03.564338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c878e-0205-4978-8438-e7dab81ffd69 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:03.569600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f593e-3bf6-423b-8a4c-c0102b455c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:03.570556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f593e-3bf6-423b-8a4c-c0102b455c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:03.570556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f593e-3bf6-423b-8a4c-c0102b455c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:03.570787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f593e-3bf6-423b-8a4c-c0102b455c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:03.571547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f593e-3bf6-423b-8a4c-c0102b455c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:03.574289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f593e-3bf6-423b-8a4c-c0102b455c40 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:03.697018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1315/2631] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:03 2022] POST /identity/v3/projects => generated 365 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:03.700194 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ffce2-9734-4e53-9a8f-d1e03c1dc411 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:03.710876 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ffce2-9734-4e53-9a8f-d1e03c1dc411 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:03.777103 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4f12-6256-4442-b2e9-db724bb12cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:03.778186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4f12-6256-4442-b2e9-db724bb12cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:03.778483 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4f12-6256-4442-b2e9-db724bb12cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:03.778755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4f12-6256-4442-b2e9-db724bb12cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:03.779400 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee4f12-6256-4442-b2e9-db724bb12cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:03.781348 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee4f12-6256-4442-b2e9-db724bb12cc7 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:03.927115 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1317/2632] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:03 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:03.931393 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f593e-3bf6-423b-8a4c-c0102b455c40 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:03.942621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f593e-3bf6-423b-8a4c-c0102b455c40 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:03.947316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97396-cda1-4370-81a8-e35f301a79ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:03.948691 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97396-cda1-4370-81a8-e35f301a79ba admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:03.949185 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97396-cda1-4370-81a8-e35f301a79ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:03.949526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97396-cda1-4370-81a8-e35f301a79ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:03.950178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a97396-cda1-4370-81a8-e35f301a79ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:03.963231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a97396-cda1-4370-81a8-e35f301a79ba admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:03.970698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1316/2633] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:03.975720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4f12-6256-4442-b2e9-db724bb12cc7 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:03.986398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4f12-6256-4442-b2e9-db724bb12cc7 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:03.991341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1dda3-5f03-4c94-bf03-14379f2e52fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:03.992007 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1dda3-5f03-4c94-bf03-14379f2e52fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:03.992284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1dda3-5f03-4c94-bf03-14379f2e52fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:03.992624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1dda3-5f03-4c94-bf03-14379f2e52fd admin admin] PATH_INFO: `/v3/projects/f74e333f31fb47a2ae7856e13ca54adf/users/49c21a5a59dc4b738f67b85d2a01accf/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:04.009773 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1dda3-5f03-4c94-bf03-14379f2e52fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74e333f31fb47a2ae7856e13ca54adf, user_id=49c21a5a59dc4b738f67b85d2a01accf, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:04.013344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1dda3-5f03-4c94-bf03-14379f2e52fd admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:04.116815 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1318/2634] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:03 2022] PUT /identity/v3/projects/f74e333f31fb47a2ae7856e13ca54adf/users/49c21a5a59dc4b738f67b85d2a01accf/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:04.126320 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97396-cda1-4370-81a8-e35f301a79ba admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:04.144716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97396-cda1-4370-81a8-e35f301a79ba admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:04.209508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58ed75-6f12-46ce-9800-74df3910004a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:04.210146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58ed75-6f12-46ce-9800-74df3910004a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:04.210483 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58ed75-6f12-46ce-9800-74df3910004a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:04.210774 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58ed75-6f12-46ce-9800-74df3910004a admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:04.211423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58ed75-6f12-46ce-9800-74df3910004a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:04.213026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58ed75-6f12-46ce-9800-74df3910004a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:04.231756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1317/2635] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:04.231959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1dda3-5f03-4c94-bf03-14379f2e52fd admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:04.239590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1dda3-5f03-4c94-bf03-14379f2e52fd admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:04.244467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7119a35-ec0c-402c-82a6-c20e4773272a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:04.245685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7119a35-ec0c-402c-82a6-c20e4773272a admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:04.246018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7119a35-ec0c-402c-82a6-c20e4773272a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:04.246454 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7119a35-ec0c-402c-82a6-c20e4773272a admin admin] PATH_INFO: `/v3/projects/f74e333f31fb47a2ae7856e13ca54adf/users/49c21a5a59dc4b738f67b85d2a01accf/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:04.248419 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7119a35-ec0c-402c-82a6-c20e4773272a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74e333f31fb47a2ae7856e13ca54adf, user_id=49c21a5a59dc4b738f67b85d2a01accf, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:04.251165 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7119a35-ec0c-402c-82a6-c20e4773272a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:04.262623 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1319/2636] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:04 2022] PUT /identity/v3/projects/f74e333f31fb47a2ae7856e13ca54adf/users/49c21a5a59dc4b738f67b85d2a01accf/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:04.276458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9980aca-13fb-4a1f-a454-5beb5953c6df None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:04.276915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9980aca-13fb-4a1f-a454-5beb5953c6df None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:04.277194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9980aca-13fb-4a1f-a454-5beb5953c6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:04.307531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a9980aca-13fb-4a1f-a454-5beb5953c6df None None] MFA Rules not processed for user `49c21a5a59dc4b738f67b85d2a01accf`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:04.309282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a9980aca-13fb-4a1f-a454-5beb5953c6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:04.357908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:04.383708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1318/2637] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:04.395396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7119a35-ec0c-402c-82a6-c20e4773272a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:04.397101 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b7119a35-ec0c-402c-82a6-c20e4773272a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:04.455353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7119a35-ec0c-402c-82a6-c20e4773272a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:04.458912 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32fa08-45c4-40b0-bb99-1aa090e33ab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:04.459514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32fa08-45c4-40b0-bb99-1aa090e33ab2 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:04.459787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32fa08-45c4-40b0-bb99-1aa090e33ab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:04.460054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32fa08-45c4-40b0-bb99-1aa090e33ab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:04.469286 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32fa08-45c4-40b0-bb99-1aa090e33ab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:04.470947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32fa08-45c4-40b0-bb99-1aa090e33ab2 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:04.481240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1320/2638] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:05.516675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9980aca-13fb-4a1f-a454-5beb5953c6df None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:05.582628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9980aca-13fb-4a1f-a454-5beb5953c6df None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:05.587290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:05.587925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:05.588268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:05.589172 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] PATH_INFO: `/v3/users/51adca6e07034efaaa8c5c35f6c3dc85` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:05.590504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51adca6e07034efaaa8c5c35f6c3dc85)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:05.592441 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:05.863756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51adca6e07034efaaa8c5c35f6c3dc85'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:05.877410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51adca6e07034efaaa8c5c35f6c3dc85'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:05.884851 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51adca6e07034efaaa8c5c35f6c3dc85'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:06.018153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51adca6e07034efaaa8c5c35f6c3dc85'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:06.018826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51adca6e07034efaaa8c5c35f6c3dc85'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:06.026097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1319/2639] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:59:05 2022] DELETE /identity/v3/users/51adca6e07034efaaa8c5c35f6c3dc85 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:06.031616 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564f396-7f11-48a8-9a16-a4c49ece4399 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:06.032513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564f396-7f11-48a8-9a16-a4c49ece4399 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:06.032513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564f396-7f11-48a8-9a16-a4c49ece4399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:06.052991 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-a564f396-7f11-48a8-9a16-a4c49ece4399 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:06.054229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a564f396-7f11-48a8-9a16-a4c49ece4399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:06.094947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:06.142772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1321/2640] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:06.159151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:06.159151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:06.229077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18162219-7cfc-4308-b181-b8dac3d11b77 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:06.234476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6d19-bf7c-4cd3-992b-06cdf178f0b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:06.236873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6d19-bf7c-4cd3-992b-06cdf178f0b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:06.236873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6d19-bf7c-4cd3-992b-06cdf178f0b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:06.236873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6d19-bf7c-4cd3-992b-06cdf178f0b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:06.250990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e6d19-bf7c-4cd3-992b-06cdf178f0b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:06.252559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e6d19-bf7c-4cd3-992b-06cdf178f0b0 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:06.263147 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1320/2641] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:06.852893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564f396-7f11-48a8-9a16-a4c49ece4399 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:06.855250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a564f396-7f11-48a8-9a16-a4c49ece4399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:06.874572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564f396-7f11-48a8-9a16-a4c49ece4399 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:06.883786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:06.884799 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:06.885326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:06.886361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] PATH_INFO: `/v3/projects/bbf8221aac334b109f409c0e68b73b67` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:06.888772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbf8221aac334b109f409c0e68b73b67)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:06.891191 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:07.029290 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbf8221aac334b109f409c0e68b73b67', 'request_id': 'req-9806ea0a-54ea-42a3-9ffa-98250ace1e64'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:07.121921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbf8221aac334b109f409c0e68b73b67', 'request_id': 'req-9806ea0a-54ea-42a3-9ffa-98250ace1e64'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:07.142507 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbf8221aac334b109f409c0e68b73b67', 'request_id': 'req-9806ea0a-54ea-42a3-9ffa-98250ace1e64'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:07.155157 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] The token cache is being invalidate because project bbf8221aac334b109f409c0e68b73b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:07.155448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:07.156131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:07.157659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1322/2642] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:06 2022] DELETE /identity/v3/projects/bbf8221aac334b109f409c0e68b73b67 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:07.161928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6d19-bf7c-4cd3-992b-06cdf178f0b0 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:07.166822 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:07.172149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6d19-bf7c-4cd3-992b-06cdf178f0b0 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:07.243087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333203c-47b8-46df-9043-cb4b7ab6fe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:07.243756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333203c-47b8-46df-9043-cb4b7ab6fe1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:07.244059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333203c-47b8-46df-9043-cb4b7ab6fe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:07.244334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333203c-47b8-46df-9043-cb4b7ab6fe1e admin admin] PATH_INFO: `/v3/users/49c21a5a59dc4b738f67b85d2a01accf` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:07.246036 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333203c-47b8-46df-9043-cb4b7ab6fe1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c21a5a59dc4b738f67b85d2a01accf)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:07.248074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333203c-47b8-46df-9043-cb4b7ab6fe1e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:07.483568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1333203c-47b8-46df-9043-cb4b7ab6fe1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c21a5a59dc4b738f67b85d2a01accf'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:07.498836 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1333203c-47b8-46df-9043-cb4b7ab6fe1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c21a5a59dc4b738f67b85d2a01accf'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:07.509889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1333203c-47b8-46df-9043-cb4b7ab6fe1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c21a5a59dc4b738f67b85d2a01accf'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:07.607203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1333203c-47b8-46df-9043-cb4b7ab6fe1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c21a5a59dc4b738f67b85d2a01accf'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:07.607848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1333203c-47b8-46df-9043-cb4b7ab6fe1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c21a5a59dc4b738f67b85d2a01accf'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:07.615087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1321/2643] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:59:07 2022] DELETE /identity/v3/users/49c21a5a59dc4b738f67b85d2a01accf => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:08.101718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:08.103988 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:08.131831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806ea0a-54ea-42a3-9ffa-98250ace1e64 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:08.190940 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:08.191576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:08.191853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:08.192124 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] PATH_INFO: `/v3/projects/f74e333f31fb47a2ae7856e13ca54adf` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:08.193261 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f74e333f31fb47a2ae7856e13ca54adf)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:08.195223 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:08.316408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f74e333f31fb47a2ae7856e13ca54adf', 'request_id': 'req-160b7012-e03e-4c34-88b7-4e2521844177'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:08.419355 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f74e333f31fb47a2ae7856e13ca54adf', 'request_id': 'req-160b7012-e03e-4c34-88b7-4e2521844177'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:08.424478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:08.426839 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f74e333f31fb47a2ae7856e13ca54adf', 'request_id': 'req-160b7012-e03e-4c34-88b7-4e2521844177'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:08.439182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] The token cache is being invalidate because project f74e333f31fb47a2ae7856e13ca54adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:08.439531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:08.440206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-160b7012-e03e-4c34-88b7-4e2521844177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:08.441904 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1323/2644] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:08 2022] DELETE /identity/v3/projects/f74e333f31fb47a2ae7856e13ca54adf => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:08.449233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b468c3-778d-4a6f-8629-f8d8911935f9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:08.449793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b468c3-778d-4a6f-8629-f8d8911935f9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:08.449793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b468c3-778d-4a6f-8629-f8d8911935f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:08.473487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-92b468c3-778d-4a6f-8629-f8d8911935f9 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:08.474757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-92b468c3-778d-4a6f-8629-f8d8911935f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:08.539158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1322/2645] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:08.548192 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aecd96-8fcd-416d-8625-822ef57bed13 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:08.548470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aecd96-8fcd-416d-8625-822ef57bed13 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:08.548741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aecd96-8fcd-416d-8625-822ef57bed13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:08.568748 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-69aecd96-8fcd-416d-8625-822ef57bed13 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:08.570012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-69aecd96-8fcd-416d-8625-822ef57bed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:08.582201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1324/2646] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:08.586983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b468c3-778d-4a6f-8629-f8d8911935f9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:08.597991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b468c3-778d-4a6f-8629-f8d8911935f9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:08.602704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62732a3f-150a-4fea-a08c-43aa2b74d045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:08.603341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62732a3f-150a-4fea-a08c-43aa2b74d045 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:08.603631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62732a3f-150a-4fea-a08c-43aa2b74d045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:08.603904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62732a3f-150a-4fea-a08c-43aa2b74d045 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:08.605156 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62732a3f-150a-4fea-a08c-43aa2b74d045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:59:08.605464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62732a3f-150a-4fea-a08c-43aa2b74d045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:08.607121 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62732a3f-150a-4fea-a08c-43aa2b74d045 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:08.619140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1323/2647] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:59:08.624543 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aecd96-8fcd-416d-8625-822ef57bed13 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:08.633882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aecd96-8fcd-416d-8625-822ef57bed13 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:08.640722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6190ca6-c1c8-4f3c-a2c2-d758a75ac4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:08.640722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6190ca6-c1c8-4f3c-a2c2-d758a75ac4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:08.640956 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6190ca6-c1c8-4f3c-a2c2-d758a75ac4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:08.641070 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6190ca6-c1c8-4f3c-a2c2-d758a75ac4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:08.642119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6190ca6-c1c8-4f3c-a2c2-d758a75ac4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:08.643651 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6190ca6-c1c8-4f3c-a2c2-d758a75ac4b5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:09.260613 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1325/2648] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:08 2022] POST /identity/v3/projects => generated 375 bytes in 637 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:09.288452 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62732a3f-150a-4fea-a08c-43aa2b74d045 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:09.301872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62732a3f-150a-4fea-a08c-43aa2b74d045 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:09.307372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:09.365395 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0a1dd-cdf3-4d47-bc71-11253276a997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:09.366008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0a1dd-cdf3-4d47-bc71-11253276a997 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:09.366291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0a1dd-cdf3-4d47-bc71-11253276a997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:09.366602 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0a1dd-cdf3-4d47-bc71-11253276a997 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:09.367275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0a1dd-cdf3-4d47-bc71-11253276a997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:09.369213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0a1dd-cdf3-4d47-bc71-11253276a997 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:09.562365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1324/2649] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:09 2022] POST /identity/v3/users => generated 288 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:09.566631 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6190ca6-c1c8-4f3c-a2c2-d758a75ac4b5 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:09.578255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6190ca6-c1c8-4f3c-a2c2-d758a75ac4b5 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:09.583457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde16ad-3fde-47d8-8fd6-b68958dc7e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:09.584081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde16ad-3fde-47d8-8fd6-b68958dc7e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:09.584361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde16ad-3fde-47d8-8fd6-b68958dc7e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:09.584633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde16ad-3fde-47d8-8fd6-b68958dc7e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:09.585294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde16ad-3fde-47d8-8fd6-b68958dc7e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:09.586958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde16ad-3fde-47d8-8fd6-b68958dc7e4a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:09.599603 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1326/2650] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:09.604043 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0a1dd-cdf3-4d47-bc71-11253276a997 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:09.615443 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0a1dd-cdf3-4d47-bc71-11253276a997 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:09.620516 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628378dc-bd26-4c20-bcbe-f51e0d199cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:09.621446 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628378dc-bd26-4c20-bcbe-f51e0d199cef admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:09.621837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628378dc-bd26-4c20-bcbe-f51e0d199cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:09.622107 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628378dc-bd26-4c20-bcbe-f51e0d199cef admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:09.622798 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628378dc-bd26-4c20-bcbe-f51e0d199cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:09.624423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628378dc-bd26-4c20-bcbe-f51e0d199cef admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:09.636782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1325/2651] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:09.641718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde16ad-3fde-47d8-8fd6-b68958dc7e4a admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:09.653473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde16ad-3fde-47d8-8fd6-b68958dc7e4a admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:09.658673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7baeb-89db-4c11-a357-fac2b3e58491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:09.659332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7baeb-89db-4c11-a357-fac2b3e58491 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:09.659601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7baeb-89db-4c11-a357-fac2b3e58491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:09.659872 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7baeb-89db-4c11-a357-fac2b3e58491 admin admin] PATH_INFO: `/v3/projects/f3a0e452bdd2465ebea7be62fdd38747/users/8ef343acaccc4a51926392123fc78b2f/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:09.676593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7baeb-89db-4c11-a357-fac2b3e58491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a0e452bdd2465ebea7be62fdd38747, user_id=8ef343acaccc4a51926392123fc78b2f, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:09.679358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7baeb-89db-4c11-a357-fac2b3e58491 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:09.787744 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1327/2652] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:09 2022] PUT /identity/v3/projects/f3a0e452bdd2465ebea7be62fdd38747/users/8ef343acaccc4a51926392123fc78b2f/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:09.807364 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3b56e-2723-4848-88da-a722d9472b18 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:09.807722 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3b56e-2723-4848-88da-a722d9472b18 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:09.808001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3b56e-2723-4848-88da-a722d9472b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:09.840434 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0cb3b56e-2723-4848-88da-a722d9472b18 None None] MFA Rules not processed for user `8ef343acaccc4a51926392123fc78b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:09.841750 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0cb3b56e-2723-4848-88da-a722d9472b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:09.932230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1326/2653] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:09.946130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7baeb-89db-4c11-a357-fac2b3e58491 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:09.948087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-36c7baeb-89db-4c11-a357-fac2b3e58491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:10.020456 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7baeb-89db-4c11-a357-fac2b3e58491 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:10.024605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf1f2f-13e4-4602-9e5c-41d3885898ba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a59337d20e4678b8a16ac35bffbdb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:10.025365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf1f2f-13e4-4602-9e5c-41d3885898ba service glance] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:10.025644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf1f2f-13e4-4602-9e5c-41d3885898ba service glance] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:10.025922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf1f2f-13e4-4602-9e5c-41d3885898ba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:10.035911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf1f2f-13e4-4602-9e5c-41d3885898ba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:10.037370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf1f2f-13e4-4602-9e5c-41d3885898ba service glance] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:10.049954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1328/2654] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:19.129241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3b56e-2723-4848-88da-a722d9472b18 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:19.130903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0cb3b56e-2723-4848-88da-a722d9472b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:19.292173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:19.303146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3b56e-2723-4848-88da-a722d9472b18 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:19.309667 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc5533-2398-4043-9ef4-1e41e6008883 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:19.310303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc5533-2398-4043-9ef4-1e41e6008883 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:19.310621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc5533-2398-4043-9ef4-1e41e6008883 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:19.310897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc5533-2398-4043-9ef4-1e41e6008883 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:19.312915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fabc5533-2398-4043-9ef4-1e41e6008883 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:19.334505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc5533-2398-4043-9ef4-1e41e6008883 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:19.336589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc5533-2398-4043-9ef4-1e41e6008883 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:19.353455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1327/2655] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:20.043030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:DELETE=1,keystone:SELECT=57,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:23.236878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf1f2f-13e4-4602-9e5c-41d3885898ba service glance] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:23.238361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-8baf1f2f-13e4-4602-9e5c-41d3885898ba service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:23.310902 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf1f2f-13e4-4602-9e5c-41d3885898ba service glance] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:23.315666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:23.316296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:23.316580 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:23.316851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] PATH_INFO: `/v3/users/d03daee53c5444fca3aebee7c2f60d38` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:23.318177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03daee53c5444fca3aebee7c2f60d38)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:23.320593 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:23.525340 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03daee53c5444fca3aebee7c2f60d38'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:23.540355 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03daee53c5444fca3aebee7c2f60d38'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:23.548246 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03daee53c5444fca3aebee7c2f60d38'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:23.641601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03daee53c5444fca3aebee7c2f60d38'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:23.642259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03daee53c5444fca3aebee7c2f60d38'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:23.649789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1329/2656] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:59:23 2022] DELETE /identity/v3/users/d03daee53c5444fca3aebee7c2f60d38 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:23.655138 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac377af9-8c78-4ced-be75-a8eaa9a8fa96 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:23.655813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac377af9-8c78-4ced-be75-a8eaa9a8fa96 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:23.655813 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac377af9-8c78-4ced-be75-a8eaa9a8fa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:23.677146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ac377af9-8c78-4ced-be75-a8eaa9a8fa96 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:23.678867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ac377af9-8c78-4ced-be75-a8eaa9a8fa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:23.741876 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1328/2657] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:59:23.752869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:23.754336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:23.839841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3a416-ec22-4ad5-9224-baa60554d717 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:23.843788 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8000d-c42e-46e6-8ca0-7fe4e6ad0faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:23.844508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8000d-c42e-46e6-8ca0-7fe4e6ad0faa service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:23.844793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8000d-c42e-46e6-8ca0-7fe4e6ad0faa service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:23.845068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8000d-c42e-46e6-8ca0-7fe4e6ad0faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:23.856268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8000d-c42e-46e6-8ca0-7fe4e6ad0faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:23.857812 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8000d-c42e-46e6-8ca0-7fe4e6ad0faa service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:23.869511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1330/2658] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:24.277180 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac377af9-8c78-4ced-be75-a8eaa9a8fa96 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:24.278682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ac377af9-8c78-4ced-be75-a8eaa9a8fa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:24.290897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac377af9-8c78-4ced-be75-a8eaa9a8fa96 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:24.294904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9d94d-27ea-4f50-b0fb-05bf9a3d95d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:24.295513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9d94d-27ea-4f50-b0fb-05bf9a3d95d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:24.295790 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9d94d-27ea-4f50-b0fb-05bf9a3d95d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:24.296063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9d94d-27ea-4f50-b0fb-05bf9a3d95d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:24.297877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b9d9d94d-27ea-4f50-b0fb-05bf9a3d95d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:24.310495 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9d94d-27ea-4f50-b0fb-05bf9a3d95d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:24.311952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9d94d-27ea-4f50-b0fb-05bf9a3d95d0 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:24.322632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1329/2659] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:24.363898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8000d-c42e-46e6-8ca0-7fe4e6ad0faa service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:24.365539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-93b8000d-c42e-46e6-8ca0-7fe4e6ad0faa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:24.382259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:24.382259 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8000d-c42e-46e6-8ca0-7fe4e6ad0faa service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:24.389566 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:24.390513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:24.390901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:24.391217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] PATH_INFO: `/v3/projects/0c36ba46e6c54fadb6e4acd8c0c11656` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:24.393005 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c36ba46e6c54fadb6e4acd8c0c11656)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:24.395750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:24.515176 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c36ba46e6c54fadb6e4acd8c0c11656', 'request_id': 'req-c0db2082-e3eb-422b-b4c3-bfe594f0673d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:24.591083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c36ba46e6c54fadb6e4acd8c0c11656', 'request_id': 'req-c0db2082-e3eb-422b-b4c3-bfe594f0673d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:24.599015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c36ba46e6c54fadb6e4acd8c0c11656', 'request_id': 'req-c0db2082-e3eb-422b-b4c3-bfe594f0673d'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:24.612622 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] The token cache is being invalidate because project 0c36ba46e6c54fadb6e4acd8c0c11656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:24.612914 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:24.613614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:24.615407 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1331/2660] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:24 2022] DELETE /identity/v3/projects/0c36ba46e6c54fadb6e4acd8c0c11656 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:24.619596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9d94d-27ea-4f50-b0fb-05bf9a3d95d0 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:24.631401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9d94d-27ea-4f50-b0fb-05bf9a3d95d0 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:24.709437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bc192-420b-48d9-941a-6dbb48ac7efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:24.710125 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bc192-420b-48d9-941a-6dbb48ac7efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:24.710366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bc192-420b-48d9-941a-6dbb48ac7efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:24.710672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bc192-420b-48d9-941a-6dbb48ac7efc admin admin] PATH_INFO: `/v3/users/8ba82e28a63c4012a08da9e882947d33` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:24.711850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bc192-420b-48d9-941a-6dbb48ac7efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ba82e28a63c4012a08da9e882947d33)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:24.714778 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bc192-420b-48d9-941a-6dbb48ac7efc admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:24.972711 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6c5bc192-420b-48d9-941a-6dbb48ac7efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ba82e28a63c4012a08da9e882947d33'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:24.993725 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6c5bc192-420b-48d9-941a-6dbb48ac7efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ba82e28a63c4012a08da9e882947d33'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:25.002360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6c5bc192-420b-48d9-941a-6dbb48ac7efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ba82e28a63c4012a08da9e882947d33'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:25.005155 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:25.126889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6c5bc192-420b-48d9-941a-6dbb48ac7efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ba82e28a63c4012a08da9e882947d33'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:25.127628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6c5bc192-420b-48d9-941a-6dbb48ac7efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ba82e28a63c4012a08da9e882947d33'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:25.138881 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1330/2661] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:59:24 2022] DELETE /identity/v3/users/8ba82e28a63c4012a08da9e882947d33 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:25.754697 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:25.757689 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:25.775102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db2082-e3eb-422b-b4c3-bfe594f0673d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:25.863515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:25.864166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:25.864451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:25.864728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] PATH_INFO: `/v3/projects/40dc374c0d974579a11ab2da7d492a58` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:25.865974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=40dc374c0d974579a11ab2da7d492a58)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:25.867956 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:25.988558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40dc374c0d974579a11ab2da7d492a58', 'request_id': 'req-3faea183-ffd2-4580-9248-d2c8b4024c1e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:26.105425 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40dc374c0d974579a11ab2da7d492a58', 'request_id': 'req-3faea183-ffd2-4580-9248-d2c8b4024c1e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:26.122943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40dc374c0d974579a11ab2da7d492a58', 'request_id': 'req-3faea183-ffd2-4580-9248-d2c8b4024c1e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:26.139184 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] The token cache is being invalidate because project 40dc374c0d974579a11ab2da7d492a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:26.139612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:26.140412 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-3faea183-ffd2-4580-9248-d2c8b4024c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:26.142260 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1332/2662] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:25 2022] DELETE /identity/v3/projects/40dc374c0d974579a11ab2da7d492a58 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:26.149565 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180d216-633d-4d52-a25b-7336248e125f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:26.150034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180d216-633d-4d52-a25b-7336248e125f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:26.150140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180d216-633d-4d52-a25b-7336248e125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:26.199338 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2180d216-633d-4d52-a25b-7336248e125f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:26.200594 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2180d216-633d-4d52-a25b-7336248e125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:26.290276 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1331/2663] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:26.299816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c6b94-86a5-449b-9706-4636ae179a6f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:26.300237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c6b94-86a5-449b-9706-4636ae179a6f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:26.300826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c6b94-86a5-449b-9706-4636ae179a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:26.325477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f98c6b94-86a5-449b-9706-4636ae179a6f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:26.327039 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f98c6b94-86a5-449b-9706-4636ae179a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:26.342489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1333/2664] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:26.345230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180d216-633d-4d52-a25b-7336248e125f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:26.362765 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180d216-633d-4d52-a25b-7336248e125f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:26.368238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1938af-ec3a-448b-8885-a26fbacd3796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:26.369022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1938af-ec3a-448b-8885-a26fbacd3796 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:26.369303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1938af-ec3a-448b-8885-a26fbacd3796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:26.369576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1938af-ec3a-448b-8885-a26fbacd3796 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:26.371282 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1938af-ec3a-448b-8885-a26fbacd3796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:59:26.371596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1938af-ec3a-448b-8885-a26fbacd3796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:26.374194 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1938af-ec3a-448b-8885-a26fbacd3796 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:26.394821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1332/2665] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:59:26.399423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c6b94-86a5-449b-9706-4636ae179a6f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:26.427834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c6b94-86a5-449b-9706-4636ae179a6f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:26.437824 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58925e9a-655c-48a0-9164-5017bbff6040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:26.438963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58925e9a-655c-48a0-9164-5017bbff6040 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:26.439498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58925e9a-655c-48a0-9164-5017bbff6040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:26.439975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58925e9a-655c-48a0-9164-5017bbff6040 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:26.441180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58925e9a-655c-48a0-9164-5017bbff6040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:26.445071 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58925e9a-655c-48a0-9164-5017bbff6040 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:26.455883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:26.584334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1334/2666] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:26 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:26.589697 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1938af-ec3a-448b-8885-a26fbacd3796 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:26.604989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1938af-ec3a-448b-8885-a26fbacd3796 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:26.685160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61394540-347a-4c56-a2b6-129c9b53cb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:26.685160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394540-347a-4c56-a2b6-129c9b53cb49 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:26.685160 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394540-347a-4c56-a2b6-129c9b53cb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:26.685558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394540-347a-4c56-a2b6-129c9b53cb49 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:26.685985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61394540-347a-4c56-a2b6-129c9b53cb49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:26.688049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61394540-347a-4c56-a2b6-129c9b53cb49 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:26.861963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1333/2667] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:26 2022] POST /identity/v3/users => generated 292 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:26.866469 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58925e9a-655c-48a0-9164-5017bbff6040 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:26.878373 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58925e9a-655c-48a0-9164-5017bbff6040 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:26.887315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f8d1b-c671-47c1-835f-8486175005ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:26.887949 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f8d1b-c671-47c1-835f-8486175005ec admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:26.890654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f8d1b-c671-47c1-835f-8486175005ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:26.890654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f8d1b-c671-47c1-835f-8486175005ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:26.890654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f8d1b-c671-47c1-835f-8486175005ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:26.891123 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f8d1b-c671-47c1-835f-8486175005ec admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:26.902660 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1335/2668] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:26.906894 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61394540-347a-4c56-a2b6-129c9b53cb49 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:26.918103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61394540-347a-4c56-a2b6-129c9b53cb49 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:26.923003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814b191-ddef-4fda-85ee-273a23eddc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:26.923606 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814b191-ddef-4fda-85ee-273a23eddc08 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:26.923891 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814b191-ddef-4fda-85ee-273a23eddc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:26.924170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814b191-ddef-4fda-85ee-273a23eddc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:26.924817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814b191-ddef-4fda-85ee-273a23eddc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:26.926992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814b191-ddef-4fda-85ee-273a23eddc08 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:26.939781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1334/2669] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:59:26.945112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f8d1b-c671-47c1-835f-8486175005ec admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:26.955936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f8d1b-c671-47c1-835f-8486175005ec admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:26.960716 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d2d96-2ffd-4bb9-8594-31b59211683e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:26.961379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d2d96-2ffd-4bb9-8594-31b59211683e admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:26.961657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d2d96-2ffd-4bb9-8594-31b59211683e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:26.961931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d2d96-2ffd-4bb9-8594-31b59211683e admin admin] PATH_INFO: `/v3/projects/d061217b6e694e10ad4121b918a096c1/users/fd82b2a076c343778a3cb66f53500eec/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:26.980080 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d2d96-2ffd-4bb9-8594-31b59211683e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d061217b6e694e10ad4121b918a096c1, user_id=fd82b2a076c343778a3cb66f53500eec, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:26.988167 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d2d96-2ffd-4bb9-8594-31b59211683e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:27.088709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1336/2670] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:26 2022] PUT /identity/v3/projects/d061217b6e694e10ad4121b918a096c1/users/fd82b2a076c343778a3cb66f53500eec/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:27.095863 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f729b78-578e-4156-8bdc-583658993830 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:27.096542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f729b78-578e-4156-8bdc-583658993830 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:27.097232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f729b78-578e-4156-8bdc-583658993830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:27.162382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0f729b78-578e-4156-8bdc-583658993830 None None] MFA Rules not processed for user `fd82b2a076c343778a3cb66f53500eec`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:27.165718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0f729b78-578e-4156-8bdc-583658993830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:27.236465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:27.263553 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1335/2671] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:27.264573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d2d96-2ffd-4bb9-8594-31b59211683e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:27.360247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d2d96-2ffd-4bb9-8594-31b59211683e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:27.367261 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d056bf-12d1-408b-adb1-faf0d432de2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:27.367633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d056bf-12d1-408b-adb1-faf0d432de2b admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:27.367921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d056bf-12d1-408b-adb1-faf0d432de2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:27.368258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d056bf-12d1-408b-adb1-faf0d432de2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:27.368920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d056bf-12d1-408b-adb1-faf0d432de2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:27.372268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d056bf-12d1-408b-adb1-faf0d432de2b admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:27.495637 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1337/2672] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:27 2022] POST /identity/v3/projects => generated 383 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:27.500566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f729b78-578e-4156-8bdc-583658993830 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:27.513694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f729b78-578e-4156-8bdc-583658993830 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:27.576802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cee564-6c25-48db-9695-b1b568b30262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:27.577336 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cee564-6c25-48db-9695-b1b568b30262 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:27.577616 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cee564-6c25-48db-9695-b1b568b30262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:27.577891 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cee564-6c25-48db-9695-b1b568b30262 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:27.578564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cee564-6c25-48db-9695-b1b568b30262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:27.580535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cee564-6c25-48db-9695-b1b568b30262 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:27.733213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1336/2673] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:27 2022] POST /identity/v3/users => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:59:27.742996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d056bf-12d1-408b-adb1-faf0d432de2b admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:27.759642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d056bf-12d1-408b-adb1-faf0d432de2b admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:27.767678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b4995-2b3a-41e4-bebe-cfebccc4de44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:27.768675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b4995-2b3a-41e4-bebe-cfebccc4de44 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:27.769043 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b4995-2b3a-41e4-bebe-cfebccc4de44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:27.769411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b4995-2b3a-41e4-bebe-cfebccc4de44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:27.770330 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b4995-2b3a-41e4-bebe-cfebccc4de44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:27.772532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b4995-2b3a-41e4-bebe-cfebccc4de44 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:27.786738 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1338/2674] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:27.793478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cee564-6c25-48db-9695-b1b568b30262 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:27.808316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cee564-6c25-48db-9695-b1b568b30262 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:27.821159 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b534a-663d-4ce6-86c8-bf47a41455e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:27.822404 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b534a-663d-4ce6-86c8-bf47a41455e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:27.822683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b534a-663d-4ce6-86c8-bf47a41455e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:27.822975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b534a-663d-4ce6-86c8-bf47a41455e0 admin admin] PATH_INFO: `/v3/projects/5b468ef25f9447d18055197fa0f9204e/users/0dcacb4967da44c5a4a402791e542178/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:27.849853 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b534a-663d-4ce6-86c8-bf47a41455e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b468ef25f9447d18055197fa0f9204e, user_id=0dcacb4967da44c5a4a402791e542178, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:27.853610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b534a-663d-4ce6-86c8-bf47a41455e0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:27.980454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1337/2675] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:27 2022] PUT /identity/v3/projects/5b468ef25f9447d18055197fa0f9204e/users/0dcacb4967da44c5a4a402791e542178/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:27.984809 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b4995-2b3a-41e4-bebe-cfebccc4de44 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:27.996022 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b4995-2b3a-41e4-bebe-cfebccc4de44 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:28.067728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17ec7c-fd42-43a1-b093-19b2551bce79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:28.068262 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ec7c-fd42-43a1-b093-19b2551bce79 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:28.068548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ec7c-fd42-43a1-b093-19b2551bce79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:28.068838 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ec7c-fd42-43a1-b093-19b2551bce79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:28.069501 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17ec7c-fd42-43a1-b093-19b2551bce79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:28.071177 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17ec7c-fd42-43a1-b093-19b2551bce79 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:28.083120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1339/2676] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:28.087757 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b534a-663d-4ce6-86c8-bf47a41455e0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:28.107341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b534a-663d-4ce6-86c8-bf47a41455e0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:28.114585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2e28d-72dd-4cf9-8c7a-3755112043b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:28.115300 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2e28d-72dd-4cf9-8c7a-3755112043b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:28.115585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2e28d-72dd-4cf9-8c7a-3755112043b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:28.115861 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2e28d-72dd-4cf9-8c7a-3755112043b0 admin admin] PATH_INFO: `/v3/projects/5b468ef25f9447d18055197fa0f9204e/users/0dcacb4967da44c5a4a402791e542178/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:28.118020 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2e28d-72dd-4cf9-8c7a-3755112043b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b468ef25f9447d18055197fa0f9204e, user_id=0dcacb4967da44c5a4a402791e542178, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:28.124927 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2e28d-72dd-4cf9-8c7a-3755112043b0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:28.161824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1338/2677] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:28 2022] PUT /identity/v3/projects/5b468ef25f9447d18055197fa0f9204e/users/0dcacb4967da44c5a4a402791e542178/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:28.180287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1f6e5-8164-4a54-a809-2b52294691b8 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:28.180575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1f6e5-8164-4a54-a809-2b52294691b8 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:28.180864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1f6e5-8164-4a54-a809-2b52294691b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:28.262237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-a7f1f6e5-8164-4a54-a809-2b52294691b8 None None] MFA Rules not processed for user `0dcacb4967da44c5a4a402791e542178`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:28.264150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a7f1f6e5-8164-4a54-a809-2b52294691b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:28.283405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:28.420588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1340/2678] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:28.430151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2e28d-72dd-4cf9-8c7a-3755112043b0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:28.434745 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-12e2e28d-72dd-4cf9-8c7a-3755112043b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:28.629688 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2e28d-72dd-4cf9-8c7a-3755112043b0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:28.640669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d6bbf-ca63-48e6-9177-0f3b94eec741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:28.641907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d6bbf-ca63-48e6-9177-0f3b94eec741 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:28.642378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d6bbf-ca63-48e6-9177-0f3b94eec741 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:28.642998 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d6bbf-ca63-48e6-9177-0f3b94eec741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:28.715762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:28.775762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d6bbf-ca63-48e6-9177-0f3b94eec741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:28.780359 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d6bbf-ca63-48e6-9177-0f3b94eec741 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:28.806632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1339/2679] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:29.181920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1f6e5-8164-4a54-a809-2b52294691b8 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:29.185013 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a7f1f6e5-8164-4a54-a809-2b52294691b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:29.267845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1f6e5-8164-4a54-a809-2b52294691b8 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:29.273153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:29.273674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:29.273948 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:29.274216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] PATH_INFO: `/v3/users/90b2c6a74c29443b94d1577b39c7ce0b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:29.275494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b2c6a74c29443b94d1577b39c7ce0b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:29.278253 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:29.540068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b2c6a74c29443b94d1577b39c7ce0b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:29.553830 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b2c6a74c29443b94d1577b39c7ce0b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:29.561485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b2c6a74c29443b94d1577b39c7ce0b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:29.642002 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b2c6a74c29443b94d1577b39c7ce0b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:29.642760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b2c6a74c29443b94d1577b39c7ce0b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:29.649451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1341/2680] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:59:29 2022] DELETE /identity/v3/users/90b2c6a74c29443b94d1577b39c7ce0b => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:29.975989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d6bbf-ca63-48e6-9177-0f3b94eec741 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:30.001496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d6bbf-ca63-48e6-9177-0f3b94eec741 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:30.078057 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:30.078744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:30.079019 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:30.079287 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] PATH_INFO: `/v3/projects/3fd4107165644976b1d8cdbeec3c8711` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:30.080463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd4107165644976b1d8cdbeec3c8711)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:30.082392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:30.245711 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd4107165644976b1d8cdbeec3c8711', 'request_id': 'req-e3039a88-c211-409c-b155-82330df53ef5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:30.252458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd4107165644976b1d8cdbeec3c8711', 'request_id': 'req-e3039a88-c211-409c-b155-82330df53ef5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:30.367268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd4107165644976b1d8cdbeec3c8711', 'request_id': 'req-e3039a88-c211-409c-b155-82330df53ef5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:30.367627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:30.369562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:30.387700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa9e27-ca09-43f1-b1dc-0d57df2075fa admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:30.389353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] The token cache is being invalidate because project 3fd4107165644976b1d8cdbeec3c8711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:30.389946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:30.391130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3039a88-c211-409c-b155-82330df53ef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:30.393588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1340/2681] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:29 2022] DELETE /identity/v3/projects/3fd4107165644976b1d8cdbeec3c8711 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:30.399432 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae51b84-d255-4253-a410-6ce4a5b0bb83 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:30.399882 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae51b84-d255-4253-a410-6ce4a5b0bb83 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:30.400352 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae51b84-d255-4253-a410-6ce4a5b0bb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:30.431998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:30.433403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-aae51b84-d255-4253-a410-6ce4a5b0bb83 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:30.434904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-aae51b84-d255-4253-a410-6ce4a5b0bb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:30.485058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828d0ff-0906-467c-836d-b545b562934b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:30.485624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828d0ff-0906-467c-836d-b545b562934b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:30.485895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828d0ff-0906-467c-836d-b545b562934b admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:30.486168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828d0ff-0906-467c-836d-b545b562934b admin admin] PATH_INFO: `/v3/users/8ef343acaccc4a51926392123fc78b2f` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:30.487413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828d0ff-0906-467c-836d-b545b562934b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef343acaccc4a51926392123fc78b2f)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:30.489444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828d0ff-0906-467c-836d-b545b562934b admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:30.519771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1341/2682] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:59:30.530761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156a09f-de83-4c0e-9882-ef3ae22b38d5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:30.531119 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156a09f-de83-4c0e-9882-ef3ae22b38d5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:30.531596 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156a09f-de83-4c0e-9882-ef3ae22b38d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:30.548454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:30.555487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b156a09f-de83-4c0e-9882-ef3ae22b38d5 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:30.556773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b156a09f-de83-4c0e-9882-ef3ae22b38d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:30.567316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1342/2683] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:59:30.571743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156a09f-de83-4c0e-9882-ef3ae22b38d5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:30.582699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156a09f-de83-4c0e-9882-ef3ae22b38d5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:30.587556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fb51e-4aa6-4478-a93d-ca78a7301000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:30.588181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fb51e-4aa6-4478-a93d-ca78a7301000 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:30.588437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fb51e-4aa6-4478-a93d-ca78a7301000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:30.589221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fb51e-4aa6-4478-a93d-ca78a7301000 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:30.589956 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fb51e-4aa6-4478-a93d-ca78a7301000 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:59:30.590486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fb51e-4aa6-4478-a93d-ca78a7301000 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:30.592178 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fb51e-4aa6-4478-a93d-ca78a7301000 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:30.606058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1343/2684] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:59:30.614577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fb51e-4aa6-4478-a93d-ca78a7301000 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:30.625242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fb51e-4aa6-4478-a93d-ca78a7301000 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:30.630498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50409ca-5fac-4cce-aa7f-08ac32aaae37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:30.637935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50409ca-5fac-4cce-aa7f-08ac32aaae37 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:30.637935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50409ca-5fac-4cce-aa7f-08ac32aaae37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:30.637935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50409ca-5fac-4cce-aa7f-08ac32aaae37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:30.637935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50409ca-5fac-4cce-aa7f-08ac32aaae37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:30.637935 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50409ca-5fac-4cce-aa7f-08ac32aaae37 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:30.719770 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e828d0ff-0906-467c-836d-b545b562934b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef343acaccc4a51926392123fc78b2f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:30.734379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e828d0ff-0906-467c-836d-b545b562934b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef343acaccc4a51926392123fc78b2f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:30.742112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e828d0ff-0906-467c-836d-b545b562934b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef343acaccc4a51926392123fc78b2f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:30.752782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1344/2685] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:30 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:59:30.757674 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50409ca-5fac-4cce-aa7f-08ac32aaae37 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:30.828358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50409ca-5fac-4cce-aa7f-08ac32aaae37 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:30.833200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd5bae-6aa0-485b-aad0-f01e5ca05478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:30.833810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd5bae-6aa0-485b-aad0-f01e5ca05478 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:30.834094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd5bae-6aa0-485b-aad0-f01e5ca05478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:30.834396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd5bae-6aa0-485b-aad0-f01e5ca05478 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:30.835067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd5bae-6aa0-485b-aad0-f01e5ca05478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:30.837010 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd5bae-6aa0-485b-aad0-f01e5ca05478 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:30.852858 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e828d0ff-0906-467c-836d-b545b562934b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef343acaccc4a51926392123fc78b2f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:30.853576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-e828d0ff-0906-467c-836d-b545b562934b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef343acaccc4a51926392123fc78b2f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:30.861523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1342/2686] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:59:30 2022] DELETE /identity/v3/users/8ef343acaccc4a51926392123fc78b2f => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:30.865445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f278fc5-5c61-45c6-beb1-7b58cd1f87ff None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:30.865726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f278fc5-5c61-45c6-beb1-7b58cd1f87ff None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:30.865997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f278fc5-5c61-45c6-beb1-7b58cd1f87ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:30.895406 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-5f278fc5-5c61-45c6-beb1-7b58cd1f87ff None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:30.896542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5f278fc5-5c61-45c6-beb1-7b58cd1f87ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:30.958029 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1345/2687] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:30 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:59:30.961999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1343/2688] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:59:30.967550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd5bae-6aa0-485b-aad0-f01e5ca05478 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:30.967550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-71cd5bae-6aa0-485b-aad0-f01e5ca05478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:30.978747 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f278fc5-5c61-45c6-beb1-7b58cd1f87ff None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:30.980883 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5f278fc5-5c61-45c6-beb1-7b58cd1f87ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:30.983157 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd5bae-6aa0-485b-aad0-f01e5ca05478 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:30.990129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005e84b-1831-4b04-bf29-363f4a184567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:30.990926 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005e84b-1831-4b04-bf29-363f4a184567 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:30.991315 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005e84b-1831-4b04-bf29-363f4a184567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:30.991617 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005e84b-1831-4b04-bf29-363f4a184567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:30.992477 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005e84b-1831-4b04-bf29-363f4a184567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:30.994056 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005e84b-1831-4b04-bf29-363f4a184567 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:31.011314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1346/2689] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:31.013743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005e84b-1831-4b04-bf29-363f4a184567 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:31.025191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005e84b-1831-4b04-bf29-363f4a184567 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:31.034541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d0d7-9a37-4723-9d1e-1fc97aa849d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:31.036052 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d0d7-9a37-4723-9d1e-1fc97aa849d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:31.037588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d0d7-9a37-4723-9d1e-1fc97aa849d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:31.037908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d0d7-9a37-4723-9d1e-1fc97aa849d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:31.038645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354d0d7-9a37-4723-9d1e-1fc97aa849d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:31.040898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354d0d7-9a37-4723-9d1e-1fc97aa849d5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:31.053925 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f278fc5-5c61-45c6-beb1-7b58cd1f87ff None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:31.062232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af074c-84ad-4571-ad63-ca5a56780e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:31.063572 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af074c-84ad-4571-ad63-ca5a56780e7c service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:31.064306 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af074c-84ad-4571-ad63-ca5a56780e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:31.064823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af074c-84ad-4571-ad63-ca5a56780e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:31.068408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1347/2690] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:31.076693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:31.076837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d0d7-9a37-4723-9d1e-1fc97aa849d5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:31.083116 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af074c-84ad-4571-ad63-ca5a56780e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:31.085748 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af074c-84ad-4571-ad63-ca5a56780e7c service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:31.094323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d0d7-9a37-4723-9d1e-1fc97aa849d5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:31.102257 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852acf6-c2f7-45a4-812a-60e47a27f6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:31.103497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852acf6-c2f7-45a4-812a-60e47a27f6df admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:31.103888 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852acf6-c2f7-45a4-812a-60e47a27f6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:31.105063 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852acf6-c2f7-45a4-812a-60e47a27f6df admin admin] PATH_INFO: `/v3/projects/bc4234ae4468419caa7f6860578a0602/users/6dae8438665643beb106ac76bec1d98f/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:31.110928 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1344/2691] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:31.129317 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852acf6-c2f7-45a4-812a-60e47a27f6df admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4234ae4468419caa7f6860578a0602, user_id=6dae8438665643beb106ac76bec1d98f, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:31.135223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852acf6-c2f7-45a4-812a-60e47a27f6df admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:31.253524 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1348/2692] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:31 2022] PUT /identity/v3/projects/bc4234ae4468419caa7f6860578a0602/users/6dae8438665643beb106ac76bec1d98f/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:31.265857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f529c27-dc41-4060-a481-fe7077306b7e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:31.266266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f529c27-dc41-4060-a481-fe7077306b7e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:31.266434 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f529c27-dc41-4060-a481-fe7077306b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:31.295355 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-2f529c27-dc41-4060-a481-fe7077306b7e None None] MFA Rules not processed for user `6dae8438665643beb106ac76bec1d98f`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:31.296625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2f529c27-dc41-4060-a481-fe7077306b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:31.383585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1345/2693] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:31.400713 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852acf6-c2f7-45a4-812a-60e47a27f6df admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:31.403577 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7852acf6-c2f7-45a4-812a-60e47a27f6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:31.522610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:31.563989 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852acf6-c2f7-45a4-812a-60e47a27f6df admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:31.572283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c15c0-47a5-465b-97e4-b325a9cb62da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:31.572995 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c15c0-47a5-465b-97e4-b325a9cb62da service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:31.573878 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c15c0-47a5-465b-97e4-b325a9cb62da service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:31.574125 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c15c0-47a5-465b-97e4-b325a9cb62da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:31.585049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c15c0-47a5-465b-97e4-b325a9cb62da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:31.588024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c15c0-47a5-465b-97e4-b325a9cb62da service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:31.606928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1349/2694] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:31.676392 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f529c27-dc41-4060-a481-fe7077306b7e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:31.677804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2f529c27-dc41-4060-a481-fe7077306b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:31.796229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f529c27-dc41-4060-a481-fe7077306b7e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:31.805370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:31.806653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:31.807172 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:31.808204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] PATH_INFO: `/v3/projects/f3a0e452bdd2465ebea7be62fdd38747` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:31.816169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a0e452bdd2465ebea7be62fdd38747)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:31.816169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:31.985544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a0e452bdd2465ebea7be62fdd38747', 'request_id': 'req-73f40f7d-254b-4d23-b25a-66f8f65b65f2'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:32.086267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a0e452bdd2465ebea7be62fdd38747', 'request_id': 'req-73f40f7d-254b-4d23-b25a-66f8f65b65f2'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:32.094629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a0e452bdd2465ebea7be62fdd38747', 'request_id': 'req-73f40f7d-254b-4d23-b25a-66f8f65b65f2'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:32.115793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] The token cache is being invalidate because project f3a0e452bdd2465ebea7be62fdd38747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:32.116313 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:32.117204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-73f40f7d-254b-4d23-b25a-66f8f65b65f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:32.118955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1346/2695] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:31 2022] DELETE /identity/v3/projects/f3a0e452bdd2465ebea7be62fdd38747 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:32.125987 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45033756-3d74-443d-8415-f9d445a725d1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:32.126670 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45033756-3d74-443d-8415-f9d445a725d1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:32.127023 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45033756-3d74-443d-8415-f9d445a725d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:32.159285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-45033756-3d74-443d-8415-f9d445a725d1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:32.161386 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-45033756-3d74-443d-8415-f9d445a725d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:32.244815 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1350/2696] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:32.259262 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68bef6-bf96-4bf2-8264-d7a20d5445cc None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:32.259603 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68bef6-bf96-4bf2-8264-d7a20d5445cc None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:32.259823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68bef6-bf96-4bf2-8264-d7a20d5445cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:32.282368 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-5a68bef6-bf96-4bf2-8264-d7a20d5445cc None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:32.283672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5a68bef6-bf96-4bf2-8264-d7a20d5445cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:32.300152 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1347/2697] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:32.304847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45033756-3d74-443d-8415-f9d445a725d1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:32.315756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45033756-3d74-443d-8415-f9d445a725d1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:32.321705 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17beb1c-2ac6-415c-8ed9-654740c45a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:32.322513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17beb1c-2ac6-415c-8ed9-654740c45a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:32.322914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17beb1c-2ac6-415c-8ed9-654740c45a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:32.323408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17beb1c-2ac6-415c-8ed9-654740c45a47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:32.324232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17beb1c-2ac6-415c-8ed9-654740c45a47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:59:32.324825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17beb1c-2ac6-415c-8ed9-654740c45a47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:32.328045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17beb1c-2ac6-415c-8ed9-654740c45a47 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:32.342458 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1351/2698] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:59:32.348635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68bef6-bf96-4bf2-8264-d7a20d5445cc None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:32.353334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:32.360285 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68bef6-bf96-4bf2-8264-d7a20d5445cc None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:32.365463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbef0eb-8c92-4b9c-9ea2-a846c38adc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:32.366133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbef0eb-8c92-4b9c-9ea2-a846c38adc57 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:32.366438 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbef0eb-8c92-4b9c-9ea2-a846c38adc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:32.366726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbef0eb-8c92-4b9c-9ea2-a846c38adc57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:32.368060 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbef0eb-8c92-4b9c-9ea2-a846c38adc57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:32.371145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbef0eb-8c92-4b9c-9ea2-a846c38adc57 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:32.489053 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1348/2699] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:32 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:59:32.494356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17beb1c-2ac6-415c-8ed9-654740c45a47 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:32.514407 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17beb1c-2ac6-415c-8ed9-654740c45a47 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:32.596405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6ef8b-e2a0-4122-b716-9e1084c9bc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:32.597029 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6ef8b-e2a0-4122-b716-9e1084c9bc03 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:32.597567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6ef8b-e2a0-4122-b716-9e1084c9bc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:32.597838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6ef8b-e2a0-4122-b716-9e1084c9bc03 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:32.598646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6ef8b-e2a0-4122-b716-9e1084c9bc03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:32.600541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6ef8b-e2a0-4122-b716-9e1084c9bc03 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:32.751930 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1352/2700] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:32 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:32.757068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbef0eb-8c92-4b9c-9ea2-a846c38adc57 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:32.769921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbef0eb-8c92-4b9c-9ea2-a846c38adc57 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:32.775504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72973c31-3cdd-4c2e-949a-1c4cded57749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:32.776133 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72973c31-3cdd-4c2e-949a-1c4cded57749 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:32.777143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72973c31-3cdd-4c2e-949a-1c4cded57749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:32.777475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72973c31-3cdd-4c2e-949a-1c4cded57749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:32.778164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72973c31-3cdd-4c2e-949a-1c4cded57749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:32.779969 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72973c31-3cdd-4c2e-949a-1c4cded57749 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:32.795217 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1349/2701] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:32.799646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6ef8b-e2a0-4122-b716-9e1084c9bc03 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:32.812752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6ef8b-e2a0-4122-b716-9e1084c9bc03 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:32.818278 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e8b51-0968-44a3-985f-27d89450b916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:32.818947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e8b51-0968-44a3-985f-27d89450b916 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:32.819224 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e8b51-0968-44a3-985f-27d89450b916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:32.819493 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e8b51-0968-44a3-985f-27d89450b916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:32.820293 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e8b51-0968-44a3-985f-27d89450b916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:32.822127 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e8b51-0968-44a3-985f-27d89450b916 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:32.838146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1353/2702] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:59:32.843376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72973c31-3cdd-4c2e-949a-1c4cded57749 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:32.854789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72973c31-3cdd-4c2e-949a-1c4cded57749 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:32.860196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e63851-fde5-4009-8d18-25802e95d0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:32.861144 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e63851-fde5-4009-8d18-25802e95d0b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:32.861470 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e63851-fde5-4009-8d18-25802e95d0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:32.861835 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e63851-fde5-4009-8d18-25802e95d0b0 admin admin] PATH_INFO: `/v3/projects/6f71249706794faeb24cd0708d6d0698/users/803816a2b15d456b981bcc397a6aac17/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:32.881921 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e63851-fde5-4009-8d18-25802e95d0b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f71249706794faeb24cd0708d6d0698, user_id=803816a2b15d456b981bcc397a6aac17, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:32.885089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e63851-fde5-4009-8d18-25802e95d0b0 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:32.980180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1350/2703] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:32 2022] PUT /identity/v3/projects/6f71249706794faeb24cd0708d6d0698/users/803816a2b15d456b981bcc397a6aac17/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:32.997721 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387f750-244e-4af8-9d0e-7b56e1648e22 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:32.998097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387f750-244e-4af8-9d0e-7b56e1648e22 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:32.998753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387f750-244e-4af8-9d0e-7b56e1648e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:33.039472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d387f750-244e-4af8-9d0e-7b56e1648e22 None None] MFA Rules not processed for user `803816a2b15d456b981bcc397a6aac17`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:33.041645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d387f750-244e-4af8-9d0e-7b56e1648e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:33.073825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:33.153460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1354/2704] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:59:33.170973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e63851-fde5-4009-8d18-25802e95d0b0 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:33.173061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-28e63851-fde5-4009-8d18-25802e95d0b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:33.253709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e63851-fde5-4009-8d18-25802e95d0b0 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:33.258085 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6df684-7a87-42b4-84eb-841c71bf5f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:33.261626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6df684-7a87-42b4-84eb-841c71bf5f88 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:33.261626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6df684-7a87-42b4-84eb-841c71bf5f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:33.261626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6df684-7a87-42b4-84eb-841c71bf5f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:33.269965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6df684-7a87-42b4-84eb-841c71bf5f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:33.271474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6df684-7a87-42b4-84eb-841c71bf5f88 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:33.282503 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1351/2705] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:35.341570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387f750-244e-4af8-9d0e-7b56e1648e22 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:35.361360 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387f750-244e-4af8-9d0e-7b56e1648e22 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:35.374223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2649031-cc43-46b8-a835-f8593ee57a17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:35.375003 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2649031-cc43-46b8-a835-f8593ee57a17 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:35.375313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2649031-cc43-46b8-a835-f8593ee57a17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:35.375791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2649031-cc43-46b8-a835-f8593ee57a17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:35.379191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c2649031-cc43-46b8-a835-f8593ee57a17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:35.466901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2649031-cc43-46b8-a835-f8593ee57a17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:35.468550 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2649031-cc43-46b8-a835-f8593ee57a17 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:35.481181 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1355/2706] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:43.278512 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=49,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:44.082460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6df684-7a87-42b4-84eb-841c71bf5f88 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:44.082460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-bb6df684-7a87-42b4-84eb-841c71bf5f88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:44.301718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6df684-7a87-42b4-84eb-841c71bf5f88 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:44.306976 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:44.307639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:44.307919 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:44.308183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] PATH_INFO: `/v3/users/6dae8438665643beb106ac76bec1d98f` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:44.309426 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dae8438665643beb106ac76bec1d98f)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:44.311452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:44.493989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dae8438665643beb106ac76bec1d98f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:44.510397 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dae8438665643beb106ac76bec1d98f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:44.518959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dae8438665643beb106ac76bec1d98f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:44.591741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dae8438665643beb106ac76bec1d98f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:44.592440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dae8438665643beb106ac76bec1d98f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:44.599962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1352/2707] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:59:44 2022] DELETE /identity/v3/users/6dae8438665643beb106ac76bec1d98f => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:44.605237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b1e24-18e3-4479-8ed6-a2cc41f8bd8d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:44.605237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b1e24-18e3-4479-8ed6-a2cc41f8bd8d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:44.606106 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b1e24-18e3-4479-8ed6-a2cc41f8bd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:44.628132 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-231b1e24-18e3-4479-8ed6-a2cc41f8bd8d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:44.629459 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-231b1e24-18e3-4479-8ed6-a2cc41f8bd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:44.703001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1356/2708] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:44.718327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:44.720459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:44.817290 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923c927-9079-4c8d-8ee0-3f92e6469d47 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:44.821791 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41c1bd-78c2-4889-be32-178914d63250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:44.822540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41c1bd-78c2-4889-be32-178914d63250 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:44.823124 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41c1bd-78c2-4889-be32-178914d63250 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:44.823555 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41c1bd-78c2-4889-be32-178914d63250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:44.834141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41c1bd-78c2-4889-be32-178914d63250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:44.835626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41c1bd-78c2-4889-be32-178914d63250 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:44.848827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1353/2709] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:45.339546 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b1e24-18e3-4479-8ed6-a2cc41f8bd8d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:45.350628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-231b1e24-18e3-4479-8ed6-a2cc41f8bd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:45.360332 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b1e24-18e3-4479-8ed6-a2cc41f8bd8d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:45.365311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:45.365970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:45.366254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:45.366578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] PATH_INFO: `/v3/projects/bc4234ae4468419caa7f6860578a0602` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:45.367844 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4234ae4468419caa7f6860578a0602)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:45.369806 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:45.497510 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4234ae4468419caa7f6860578a0602', 'request_id': 'req-30706c81-faf1-435c-acd0-e78f1ea16fba'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:45.499782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:45.506222 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4234ae4468419caa7f6860578a0602', 'request_id': 'req-30706c81-faf1-435c-acd0-e78f1ea16fba'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:45.596041 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4234ae4468419caa7f6860578a0602', 'request_id': 'req-30706c81-faf1-435c-acd0-e78f1ea16fba'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:45.610191 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] The token cache is being invalidate because project bc4234ae4468419caa7f6860578a0602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:45.611155 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:45.611486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-30706c81-faf1-435c-acd0-e78f1ea16fba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:45.613091 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1357/2710] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:45 2022] DELETE /identity/v3/projects/bc4234ae4468419caa7f6860578a0602 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:45.618551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd32996-6ab7-41e1-adb0-aaeed499cc1e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:45.619179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd32996-6ab7-41e1-adb0-aaeed499cc1e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:45.619179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd32996-6ab7-41e1-adb0-aaeed499cc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:45.642153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7bd32996-6ab7-41e1-adb0-aaeed499cc1e None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:45.643509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7bd32996-6ab7-41e1-adb0-aaeed499cc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:45.665521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:45.718466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1354/2711] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:45.727561 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3138-3e39-43bb-b41b-b58a305c578c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:45.727959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3138-3e39-43bb-b41b-b58a305c578c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:45.728280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3138-3e39-43bb-b41b-b58a305c578c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:45.751109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f41f3138-3e39-43bb-b41b-b58a305c578c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:45.752376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f41f3138-3e39-43bb-b41b-b58a305c578c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:45.761727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1358/2712] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:45.766568 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd32996-6ab7-41e1-adb0-aaeed499cc1e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:45.777559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd32996-6ab7-41e1-adb0-aaeed499cc1e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:45.782307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b815679-6bdb-4a3e-a7d7-4395cf1a2d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:45.783018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b815679-6bdb-4a3e-a7d7-4395cf1a2d74 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:45.783287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b815679-6bdb-4a3e-a7d7-4395cf1a2d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:45.783618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b815679-6bdb-4a3e-a7d7-4395cf1a2d74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:45.784234 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b815679-6bdb-4a3e-a7d7-4395cf1a2d74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:59:45.784540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b815679-6bdb-4a3e-a7d7-4395cf1a2d74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:45.786198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b815679-6bdb-4a3e-a7d7-4395cf1a2d74 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:45.800045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1355/2713] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:59:45.804966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f3138-3e39-43bb-b41b-b58a305c578c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:45.816763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f3138-3e39-43bb-b41b-b58a305c578c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:45.824588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ede4d6-8c33-4948-af00-e7aae32759d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:45.825793 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ede4d6-8c33-4948-af00-e7aae32759d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:45.826241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ede4d6-8c33-4948-af00-e7aae32759d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:45.826741 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ede4d6-8c33-4948-af00-e7aae32759d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:45.827855 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ede4d6-8c33-4948-af00-e7aae32759d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:45.831699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ede4d6-8c33-4948-af00-e7aae32759d7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:45.958958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1359/2714] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:45 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:45.964128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b815679-6bdb-4a3e-a7d7-4395cf1a2d74 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:45.975633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b815679-6bdb-4a3e-a7d7-4395cf1a2d74 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:46.044635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294216e5-6621-416f-a0bd-afdd0b0aa759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:46.045256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294216e5-6621-416f-a0bd-afdd0b0aa759 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:46.045534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294216e5-6621-416f-a0bd-afdd0b0aa759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:46.045801 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294216e5-6621-416f-a0bd-afdd0b0aa759 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:46.046492 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294216e5-6621-416f-a0bd-afdd0b0aa759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:46.048475 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294216e5-6621-416f-a0bd-afdd0b0aa759 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:46.420329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1356/2715] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:45 2022] POST /identity/v3/users => generated 290 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:59:46.424582 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ede4d6-8c33-4948-af00-e7aae32759d7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:46.435539 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ede4d6-8c33-4948-af00-e7aae32759d7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:46.440911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8588bb-1a93-49dc-9766-e7ca9d74995a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:46.441615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8588bb-1a93-49dc-9766-e7ca9d74995a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:46.441948 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8588bb-1a93-49dc-9766-e7ca9d74995a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:46.442272 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8588bb-1a93-49dc-9766-e7ca9d74995a admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:46.443032 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8588bb-1a93-49dc-9766-e7ca9d74995a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:46.445038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8588bb-1a93-49dc-9766-e7ca9d74995a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:46.458998 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1360/2716] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:46.463356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294216e5-6621-416f-a0bd-afdd0b0aa759 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:46.476942 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294216e5-6621-416f-a0bd-afdd0b0aa759 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:46.483061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d04aa-d198-473b-b88f-71dd5462bdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:46.483678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d04aa-d198-473b-b88f-71dd5462bdc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:46.483954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d04aa-d198-473b-b88f-71dd5462bdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:46.484216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d04aa-d198-473b-b88f-71dd5462bdc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:46.484865 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d04aa-d198-473b-b88f-71dd5462bdc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:46.486702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d04aa-d198-473b-b88f-71dd5462bdc5 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:46.501255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1357/2717] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:46.506254 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8588bb-1a93-49dc-9766-e7ca9d74995a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:46.517198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8588bb-1a93-49dc-9766-e7ca9d74995a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:46.523997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4353d92-86eb-435a-a5b6-71fdf83720e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:46.523997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4353d92-86eb-435a-a5b6-71fdf83720e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:46.523997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4353d92-86eb-435a-a5b6-71fdf83720e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:46.523997 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4353d92-86eb-435a-a5b6-71fdf83720e5 admin admin] PATH_INFO: `/v3/projects/b86fe5a13c7e41c787fe6e8ab44d3b73/users/44a591ea1e4846b2b248b00f808d27f0/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:46.547486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4353d92-86eb-435a-a5b6-71fdf83720e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b86fe5a13c7e41c787fe6e8ab44d3b73, user_id=44a591ea1e4846b2b248b00f808d27f0, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:46.550225 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4353d92-86eb-435a-a5b6-71fdf83720e5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:46.660955 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1361/2718] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:46 2022] PUT /identity/v3/projects/b86fe5a13c7e41c787fe6e8ab44d3b73/users/44a591ea1e4846b2b248b00f808d27f0/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:46.672358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8e718-5c47-4850-99c2-67e74e452ac1 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:46.672962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8e718-5c47-4850-99c2-67e74e452ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:46.672962 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8e718-5c47-4850-99c2-67e74e452ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:46.705143 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-4ec8e718-5c47-4850-99c2-67e74e452ac1 None None] MFA Rules not processed for user `44a591ea1e4846b2b248b00f808d27f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:46.706449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4ec8e718-5c47-4850-99c2-67e74e452ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:46.808845 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1358/2719] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:46.814365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4353d92-86eb-435a-a5b6-71fdf83720e5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:46.891206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4353d92-86eb-435a-a5b6-71fdf83720e5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:46.897804 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8267029-7bcf-4ded-81cf-d2fed4248987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:46.899093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267029-7bcf-4ded-81cf-d2fed4248987 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:46.899692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267029-7bcf-4ded-81cf-d2fed4248987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:46.900200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267029-7bcf-4ded-81cf-d2fed4248987 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:46.902873 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8267029-7bcf-4ded-81cf-d2fed4248987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:46.906921 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8267029-7bcf-4ded-81cf-d2fed4248987 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:47.032846 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1362/2720] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:46 2022] POST /identity/v3/projects => generated 381 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:47.037893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8e718-5c47-4850-99c2-67e74e452ac1 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:47.055801 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8e718-5c47-4850-99c2-67e74e452ac1 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:47.072364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:47.129440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b959a32-4230-4087-b3e2-9a9e237d1448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:47.130056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b959a32-4230-4087-b3e2-9a9e237d1448 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:47.130335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b959a32-4230-4087-b3e2-9a9e237d1448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:47.130642 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b959a32-4230-4087-b3e2-9a9e237d1448 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:47.131294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b959a32-4230-4087-b3e2-9a9e237d1448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:47.133418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b959a32-4230-4087-b3e2-9a9e237d1448 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:47.253009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1359/2721] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:47 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:47.258624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8267029-7bcf-4ded-81cf-d2fed4248987 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:47.271925 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8267029-7bcf-4ded-81cf-d2fed4248987 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:47.277953 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65c334-84b2-4d02-b401-63fdb1361824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:47.278635 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65c334-84b2-4d02-b401-63fdb1361824 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:47.278967 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65c334-84b2-4d02-b401-63fdb1361824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:47.279189 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65c334-84b2-4d02-b401-63fdb1361824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:47.279943 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65c334-84b2-4d02-b401-63fdb1361824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:47.282376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65c334-84b2-4d02-b401-63fdb1361824 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:47.297869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1363/2722] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:59:47.304342 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b959a32-4230-4087-b3e2-9a9e237d1448 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:47.320594 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b959a32-4230-4087-b3e2-9a9e237d1448 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:47.329382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc5376-0539-4b29-8ccf-df6e1575d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:47.330571 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc5376-0539-4b29-8ccf-df6e1575d251 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:47.331067 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc5376-0539-4b29-8ccf-df6e1575d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:47.331558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc5376-0539-4b29-8ccf-df6e1575d251 admin admin] PATH_INFO: `/v3/projects/38db15f31dca445e8defb77b3f0c5770/users/4b5f416ac07f40d9aad0438385496899/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:47.360947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc5376-0539-4b29-8ccf-df6e1575d251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38db15f31dca445e8defb77b3f0c5770, user_id=4b5f416ac07f40d9aad0438385496899, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:47.372086 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc5376-0539-4b29-8ccf-df6e1575d251 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:47.519595 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1360/2723] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:47 2022] PUT /identity/v3/projects/38db15f31dca445e8defb77b3f0c5770/users/4b5f416ac07f40d9aad0438385496899/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:47.523886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65c334-84b2-4d02-b401-63fdb1361824 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:47.536630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65c334-84b2-4d02-b401-63fdb1361824 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:47.601501 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:47.617132 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839565c9-561d-43f8-9e66-44d30b45d470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:47.618152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839565c9-561d-43f8-9e66-44d30b45d470 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:47.618698 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839565c9-561d-43f8-9e66-44d30b45d470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:47.619179 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839565c9-561d-43f8-9e66-44d30b45d470 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:47.621024 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839565c9-561d-43f8-9e66-44d30b45d470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:47.624102 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839565c9-561d-43f8-9e66-44d30b45d470 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:47.641423 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1364/2724] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:47.645505 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc5376-0539-4b29-8ccf-df6e1575d251 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:47.657487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc5376-0539-4b29-8ccf-df6e1575d251 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:47.664125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be85dd-5e20-473e-a863-761cc0e072b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:47.664812 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be85dd-5e20-473e-a863-761cc0e072b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:47.665092 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be85dd-5e20-473e-a863-761cc0e072b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:47.665401 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be85dd-5e20-473e-a863-761cc0e072b6 admin admin] PATH_INFO: `/v3/projects/38db15f31dca445e8defb77b3f0c5770/users/4b5f416ac07f40d9aad0438385496899/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:47.668094 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be85dd-5e20-473e-a863-761cc0e072b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38db15f31dca445e8defb77b3f0c5770, user_id=4b5f416ac07f40d9aad0438385496899, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:47.671522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be85dd-5e20-473e-a863-761cc0e072b6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:47.685146 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1361/2725] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:47 2022] PUT /identity/v3/projects/38db15f31dca445e8defb77b3f0c5770/users/4b5f416ac07f40d9aad0438385496899/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:47.697807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88c849-6297-4e37-b134-2c5efa2dc7b9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:47.698970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88c849-6297-4e37-b134-2c5efa2dc7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:47.699291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88c849-6297-4e37-b134-2c5efa2dc7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:47.867567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7a88c849-6297-4e37-b134-2c5efa2dc7b9 None None] MFA Rules not processed for user `4b5f416ac07f40d9aad0438385496899`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:47.869312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7a88c849-6297-4e37-b134-2c5efa2dc7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:47.965762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1365/2726] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:47.979676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be85dd-5e20-473e-a863-761cc0e072b6 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:47.981602 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-64be85dd-5e20-473e-a863-761cc0e072b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:48.052281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be85dd-5e20-473e-a863-761cc0e072b6 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:48.056375 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb09c6e-5568-4baa-89e9-1870fb0b115f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:48.057137 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb09c6e-5568-4baa-89e9-1870fb0b115f service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:48.057981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb09c6e-5568-4baa-89e9-1870fb0b115f service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:48.058305 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb09c6e-5568-4baa-89e9-1870fb0b115f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:48.071448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb09c6e-5568-4baa-89e9-1870fb0b115f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:48.075364 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb09c6e-5568-4baa-89e9-1870fb0b115f service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:48.095315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1362/2727] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:48.253264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88c849-6297-4e37-b134-2c5efa2dc7b9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:48.255048 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7a88c849-6297-4e37-b134-2c5efa2dc7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:48.335884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88c849-6297-4e37-b134-2c5efa2dc7b9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:48.341172 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:48.341919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:48.342202 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:48.342527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] PATH_INFO: `/v3/users/fd82b2a076c343778a3cb66f53500eec` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:48.343866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd82b2a076c343778a3cb66f53500eec)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:48.345862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:48.376686 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb09c6e-5568-4baa-89e9-1870fb0b115f service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:48.388786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb09c6e-5568-4baa-89e9-1870fb0b115f service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:48.393644 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2cf43-f51d-4bb8-a790-83bcc3822e72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:48.394449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2cf43-f51d-4bb8-a790-83bcc3822e72 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:48.394822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2cf43-f51d-4bb8-a790-83bcc3822e72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:48.395210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2cf43-f51d-4bb8-a790-83bcc3822e72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:48.465415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2cf43-f51d-4bb8-a790-83bcc3822e72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:48.467164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2cf43-f51d-4bb8-a790-83bcc3822e72 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:48.478715 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1363/2728] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:59:48.638762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd82b2a076c343778a3cb66f53500eec'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:48.657340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd82b2a076c343778a3cb66f53500eec'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:48.665664 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd82b2a076c343778a3cb66f53500eec'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:48.668413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:48.763165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd82b2a076c343778a3cb66f53500eec'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:48.763857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd82b2a076c343778a3cb66f53500eec'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:48.774643 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1366/2729] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:59:48 2022] DELETE /identity/v3/users/fd82b2a076c343778a3cb66f53500eec => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:48.781015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1c154-b1c2-497d-bc94-b5ce557dd9e7 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:48.781597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1c154-b1c2-497d-bc94-b5ce557dd9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:48.781597 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1c154-b1c2-497d-bc94-b5ce557dd9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:48.809410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-96e1c154-b1c2-497d-bc94-b5ce557dd9e7 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:48.811834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-96e1c154-b1c2-497d-bc94-b5ce557dd9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:48.821061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:48.910997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1364/2730] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:48.925658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:48.927283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:49.008700 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2d213-fecc-4f09-94a5-2af78cfa4a3f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:49.013689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11cafd-7ead-4638-968f-2efdf0b02b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:49.014399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11cafd-7ead-4638-968f-2efdf0b02b69 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:49.014715 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11cafd-7ead-4638-968f-2efdf0b02b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:49.014992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11cafd-7ead-4638-968f-2efdf0b02b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:49.025767 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11cafd-7ead-4638-968f-2efdf0b02b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:49.028174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11cafd-7ead-4638-968f-2efdf0b02b69 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:49.039454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1367/2731] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:49.599895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1c154-b1c2-497d-bc94-b5ce557dd9e7 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:49.601627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-96e1c154-b1c2-497d-bc94-b5ce557dd9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:49.618236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1c154-b1c2-497d-bc94-b5ce557dd9e7 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:49.625567 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:49.633510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:49.633510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:49.633510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] PATH_INFO: `/v3/projects/d061217b6e694e10ad4121b918a096c1` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:49.633510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d061217b6e694e10ad4121b918a096c1)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:49.633510 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:49.754804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd061217b6e694e10ad4121b918a096c1', 'request_id': 'req-ef3b75bf-e5e6-4749-992c-dd899281ecdb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:49.893887 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd061217b6e694e10ad4121b918a096c1', 'request_id': 'req-ef3b75bf-e5e6-4749-992c-dd899281ecdb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:49.901648 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd061217b6e694e10ad4121b918a096c1', 'request_id': 'req-ef3b75bf-e5e6-4749-992c-dd899281ecdb'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:49.914552 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] The token cache is being invalidate because project d061217b6e694e10ad4121b918a096c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:49.914860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:49.915515 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:49.917525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1365/2732] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:49 2022] DELETE /identity/v3/projects/d061217b6e694e10ad4121b918a096c1 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:49.922103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11cafd-7ead-4638-968f-2efdf0b02b69 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:49.934173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11cafd-7ead-4638-968f-2efdf0b02b69 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:50.009374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdb8ff-968b-40bb-b73b-f9c588793abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:50.010378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb8ff-968b-40bb-b73b-f9c588793abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:50.010378 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb8ff-968b-40bb-b73b-f9c588793abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:50.010914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb8ff-968b-40bb-b73b-f9c588793abd admin admin] PATH_INFO: `/v3/users/0dcacb4967da44c5a4a402791e542178` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:50.012488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdb8ff-968b-40bb-b73b-f9c588793abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dcacb4967da44c5a4a402791e542178)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:50.015574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdb8ff-968b-40bb-b73b-f9c588793abd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:50.254773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-52cdb8ff-968b-40bb-b73b-f9c588793abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dcacb4967da44c5a4a402791e542178'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:50.270543 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-52cdb8ff-968b-40bb-b73b-f9c588793abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dcacb4967da44c5a4a402791e542178'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:50.279548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-52cdb8ff-968b-40bb-b73b-f9c588793abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dcacb4967da44c5a4a402791e542178'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:50.367704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-52cdb8ff-968b-40bb-b73b-f9c588793abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dcacb4967da44c5a4a402791e542178'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:50.368434 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-52cdb8ff-968b-40bb-b73b-f9c588793abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dcacb4967da44c5a4a402791e542178'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:50.377923 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1368/2733] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:59:49 2022] DELETE /identity/v3/users/0dcacb4967da44c5a4a402791e542178 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:51.271076 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:51.272511 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:51.286142 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b75bf-e5e6-4749-992c-dd899281ecdb admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:51.354408 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:51.355514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:51.355966 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:51.356449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] PATH_INFO: `/v3/projects/5b468ef25f9447d18055197fa0f9204e` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:51.358563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b468ef25f9447d18055197fa0f9204e)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:51.362313 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:51.537296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:51.554231 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b468ef25f9447d18055197fa0f9204e', 'request_id': 'req-4c39cf77-afc1-4f08-8cde-df0812ee3a23'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:51.688810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b468ef25f9447d18055197fa0f9204e', 'request_id': 'req-4c39cf77-afc1-4f08-8cde-df0812ee3a23'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:51.703436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b468ef25f9447d18055197fa0f9204e', 'request_id': 'req-4c39cf77-afc1-4f08-8cde-df0812ee3a23'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:51.718578 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] The token cache is being invalidate because project 5b468ef25f9447d18055197fa0f9204e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:51.719064 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:51.720149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-4c39cf77-afc1-4f08-8cde-df0812ee3a23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:51.722618 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1366/2734] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:51 2022] DELETE /identity/v3/projects/5b468ef25f9447d18055197fa0f9204e => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:51.731862 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f8ceb-fb7f-4d7a-a67e-ee82154ee4c2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:51.732470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f8ceb-fb7f-4d7a-a67e-ee82154ee4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:51.732470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f8ceb-fb7f-4d7a-a67e-ee82154ee4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:51.753798 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-590f8ceb-fb7f-4d7a-a67e-ee82154ee4c2 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:51.755086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-590f8ceb-fb7f-4d7a-a67e-ee82154ee4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:51.765490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:51.837684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1369/2735] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:51.848293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68376e-7ac2-45af-a804-695f5eff5b40 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:51.848836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68376e-7ac2-45af-a804-695f5eff5b40 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:51.849252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68376e-7ac2-45af-a804-695f5eff5b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:51.881482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-6e68376e-7ac2-45af-a804-695f5eff5b40 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:51.883346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6e68376e-7ac2-45af-a804-695f5eff5b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:51.896461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1367/2736] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:51.903786 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f8ceb-fb7f-4d7a-a67e-ee82154ee4c2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:51.915556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f8ceb-fb7f-4d7a-a67e-ee82154ee4c2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:51.922266 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3ae06-ee66-4a5f-983a-4f10984a43b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:51.923168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3ae06-ee66-4a5f-983a-4f10984a43b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:51.923453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3ae06-ee66-4a5f-983a-4f10984a43b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:51.923867 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3ae06-ee66-4a5f-983a-4f10984a43b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:51.924642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3ae06-ee66-4a5f-983a-4f10984a43b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:59:51.924942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3ae06-ee66-4a5f-983a-4f10984a43b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:51.928093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3ae06-ee66-4a5f-983a-4f10984a43b4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:51.949524 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1370/2737] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:59:51.954702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68376e-7ac2-45af-a804-695f5eff5b40 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:51.965656 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68376e-7ac2-45af-a804-695f5eff5b40 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:51.971122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88efb66-ace2-43fd-8a06-582b35228325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:51.971783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88efb66-ace2-43fd-8a06-582b35228325 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:51.972104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88efb66-ace2-43fd-8a06-582b35228325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:51.972383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88efb66-ace2-43fd-8a06-582b35228325 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:51.973304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88efb66-ace2-43fd-8a06-582b35228325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:51.975494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88efb66-ace2-43fd-8a06-582b35228325 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:52.089931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1368/2738] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:51 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:52.094288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3ae06-ee66-4a5f-983a-4f10984a43b4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:52.105773 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3ae06-ee66-4a5f-983a-4f10984a43b4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:52.185763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba0033-b2d8-4818-b0b0-69727384ac2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:52.186575 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba0033-b2d8-4818-b0b0-69727384ac2c admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:52.186966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba0033-b2d8-4818-b0b0-69727384ac2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:52.187507 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba0033-b2d8-4818-b0b0-69727384ac2c admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:52.188238 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba0033-b2d8-4818-b0b0-69727384ac2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:52.191391 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba0033-b2d8-4818-b0b0-69727384ac2c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:52.311908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1371/2739] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:52 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:59:52.317180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88efb66-ace2-43fd-8a06-582b35228325 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:52.332810 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88efb66-ace2-43fd-8a06-582b35228325 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:52.340852 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e948dc-c87a-4d22-a235-e8ca17d326d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:52.341537 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e948dc-c87a-4d22-a235-e8ca17d326d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:52.341854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e948dc-c87a-4d22-a235-e8ca17d326d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:52.342267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e948dc-c87a-4d22-a235-e8ca17d326d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:52.343624 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e948dc-c87a-4d22-a235-e8ca17d326d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:52.353698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e948dc-c87a-4d22-a235-e8ca17d326d4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:52.368204 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1369/2740] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:59:52.373619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba0033-b2d8-4818-b0b0-69727384ac2c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:52.387325 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba0033-b2d8-4818-b0b0-69727384ac2c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:52.392454 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b400d0-b553-48b1-b51a-66a6a1e6d308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:52.393071 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b400d0-b553-48b1-b51a-66a6a1e6d308 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:52.393345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b400d0-b553-48b1-b51a-66a6a1e6d308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:52.393614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b400d0-b553-48b1-b51a-66a6a1e6d308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:52.396165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b400d0-b553-48b1-b51a-66a6a1e6d308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:52.396165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b400d0-b553-48b1-b51a-66a6a1e6d308 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:52.408717 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1372/2741] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:52.413501 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e948dc-c87a-4d22-a235-e8ca17d326d4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:52.425439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e948dc-c87a-4d22-a235-e8ca17d326d4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:52.430732 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2248a-c550-433b-ad0d-52577b0cec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:52.431462 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2248a-c550-433b-ad0d-52577b0cec42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:52.431742 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2248a-c550-433b-ad0d-52577b0cec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:52.432015 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2248a-c550-433b-ad0d-52577b0cec42 admin admin] PATH_INFO: `/v3/projects/c15bc2fcfc294b3ab91cd1c2d8f2dcbc/users/993c4d95a7fc4bb0b910033fa3f7409d/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:52.451636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2248a-c550-433b-ad0d-52577b0cec42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15bc2fcfc294b3ab91cd1c2d8f2dcbc, user_id=993c4d95a7fc4bb0b910033fa3f7409d, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:52.455097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2248a-c550-433b-ad0d-52577b0cec42 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:52.608946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1370/2742] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:52 2022] PUT /identity/v3/projects/c15bc2fcfc294b3ab91cd1c2d8f2dcbc/users/993c4d95a7fc4bb0b910033fa3f7409d/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:52.624031 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b00fd-89c5-430f-9e77-32e100041ca9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:52.624315 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b00fd-89c5-430f-9e77-32e100041ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:52.624588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b00fd-89c5-430f-9e77-32e100041ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:52.655875 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-220b00fd-89c5-430f-9e77-32e100041ca9 None None] MFA Rules not processed for user `993c4d95a7fc4bb0b910033fa3f7409d`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:52.657153 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-220b00fd-89c5-430f-9e77-32e100041ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:52.735636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1373/2743] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:52.762508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2248a-c550-433b-ad0d-52577b0cec42 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:52.764229 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-12b2248a-c550-433b-ad0d-52577b0cec42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:52.842607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2248a-c550-433b-ad0d-52577b0cec42 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:52.846629 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e903d12-3c62-4032-971d-c04fe93acbdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:52.847249 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e903d12-3c62-4032-971d-c04fe93acbdb service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:52.847648 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e903d12-3c62-4032-971d-c04fe93acbdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:52.848150 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e903d12-3c62-4032-971d-c04fe93acbdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:52.859577 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e903d12-3c62-4032-971d-c04fe93acbdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:52.861834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e903d12-3c62-4032-971d-c04fe93acbdb service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:52.878521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1371/2744] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:52.895100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b00fd-89c5-430f-9e77-32e100041ca9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:52.907134 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:52.965357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b00fd-89c5-430f-9e77-32e100041ca9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:52.970476 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:52.971340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:52.971718 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:52.972087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] PATH_INFO: `/v3/users/993c4d95a7fc4bb0b910033fa3f7409d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:52.973630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=993c4d95a7fc4bb0b910033fa3f7409d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:52.975678 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:53.223223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993c4d95a7fc4bb0b910033fa3f7409d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:53.238313 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993c4d95a7fc4bb0b910033fa3f7409d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:53.246694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993c4d95a7fc4bb0b910033fa3f7409d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:53.384499 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993c4d95a7fc4bb0b910033fa3f7409d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:53.385155 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993c4d95a7fc4bb0b910033fa3f7409d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:53.392382 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1374/2745] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Aug 29 23:59:52 2022] DELETE /identity/v3/users/993c4d95a7fc4bb0b910033fa3f7409d => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:53.397445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30adc0-04db-4f1a-b6e2-3bd7e2d4fd68 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:53.398236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30adc0-04db-4f1a-b6e2-3bd7e2d4fd68 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:53.398236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30adc0-04db-4f1a-b6e2-3bd7e2d4fd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:53.419399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7c30adc0-04db-4f1a-b6e2-3bd7e2d4fd68 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:53.420610 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7c30adc0-04db-4f1a-b6e2-3bd7e2d4fd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:53.481391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1372/2746] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:53.492911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:53.494619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:53.559470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628f920-771d-46d6-aecb-66ced18e5e20 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:53.561518 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14c862-f65e-43b2-9494-113230a1e4f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:53.562045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c862-f65e-43b2-9494-113230a1e4f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:53.562319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c862-f65e-43b2-9494-113230a1e4f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:53.562620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c862-f65e-43b2-9494-113230a1e4f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:53.574646 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14c862-f65e-43b2-9494-113230a1e4f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:53.576104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14c862-f65e-43b2-9494-113230a1e4f7 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:53.586704 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1375/2747] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:59:54.153805 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30adc0-04db-4f1a-b6e2-3bd7e2d4fd68 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:54.155436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7c30adc0-04db-4f1a-b6e2-3bd7e2d4fd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:54.164898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:54.169540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30adc0-04db-4f1a-b6e2-3bd7e2d4fd68 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:54.174751 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:54.175440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:54.175723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:54.175998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] PATH_INFO: `/v3/projects/c15bc2fcfc294b3ab91cd1c2d8f2dcbc` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:54.177334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15bc2fcfc294b3ab91cd1c2d8f2dcbc)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:54.179164 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:54.305552 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15bc2fcfc294b3ab91cd1c2d8f2dcbc', 'request_id': 'req-da5ca051-282c-49d5-a000-bb354a7dc41a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:54.452400 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15bc2fcfc294b3ab91cd1c2d8f2dcbc', 'request_id': 'req-da5ca051-282c-49d5-a000-bb354a7dc41a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:54.459068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15bc2fcfc294b3ab91cd1c2d8f2dcbc', 'request_id': 'req-da5ca051-282c-49d5-a000-bb354a7dc41a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:54.472619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] The token cache is being invalidate because project c15bc2fcfc294b3ab91cd1c2d8f2dcbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:59:54.473391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:54.474558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-da5ca051-282c-49d5-a000-bb354a7dc41a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:59:54.477072 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1373/2748] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Aug 29 23:59:54 2022] DELETE /identity/v3/projects/c15bc2fcfc294b3ab91cd1c2d8f2dcbc => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:59:54.484509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf56c1-19f3-4395-9693-ff398953e16d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:54.485151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf56c1-19f3-4395-9693-ff398953e16d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:54.485151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf56c1-19f3-4395-9693-ff398953e16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:54.505568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-bfcf56c1-19f3-4395-9693-ff398953e16d None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:54.506865 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bfcf56c1-19f3-4395-9693-ff398953e16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:54.534130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:59:54.569059 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1376/2749] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:54.577919 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42280ca4-8248-46e5-bbd0-665df6dd96f1 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:54.578221 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42280ca4-8248-46e5-bbd0-665df6dd96f1 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:54.578508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42280ca4-8248-46e5-bbd0-665df6dd96f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:54.601476 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-42280ca4-8248-46e5-bbd0-665df6dd96f1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:54.604771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-42280ca4-8248-46e5-bbd0-665df6dd96f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:54.615109 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1374/2750] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:54.619290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf56c1-19f3-4395-9693-ff398953e16d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:54.630131 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf56c1-19f3-4395-9693-ff398953e16d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:54.634831 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53016-058b-4470-aa7a-1bdad06abe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:54.635472 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53016-058b-4470-aa7a-1bdad06abe4a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:54.635795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53016-058b-4470-aa7a-1bdad06abe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:54.636069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53016-058b-4470-aa7a-1bdad06abe4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:54.636717 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e53016-058b-4470-aa7a-1bdad06abe4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:59:54.637211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e53016-058b-4470-aa7a-1bdad06abe4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:54.640464 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e53016-058b-4470-aa7a-1bdad06abe4a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:54.662952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1377/2751] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Aug 29 23:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:59:54.667418 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42280ca4-8248-46e5-bbd0-665df6dd96f1 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:54.677700 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42280ca4-8248-46e5-bbd0-665df6dd96f1 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:54.682565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09a76f-c610-4c79-939c-cacb9233d3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:54.683175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09a76f-c610-4c79-939c-cacb9233d3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:54.683449 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09a76f-c610-4c79-939c-cacb9233d3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:54.683716 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09a76f-c610-4c79-939c-cacb9233d3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:54.684626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09a76f-c610-4c79-939c-cacb9233d3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:54.686592 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09a76f-c610-4c79-939c-cacb9233d3f6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:54.804787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1375/2752] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Aug 29 23:59:54 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:59:54.809619 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53016-058b-4470-aa7a-1bdad06abe4a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:54.820672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53016-058b-4470-aa7a-1bdad06abe4a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:54.882740 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ec420-10ef-40c2-ab39-42b3983114f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:54.883344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ec420-10ef-40c2-ab39-42b3983114f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:54.883640 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ec420-10ef-40c2-ab39-42b3983114f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:54.883904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ec420-10ef-40c2-ab39-42b3983114f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:54.884552 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ec420-10ef-40c2-ab39-42b3983114f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:54.886493 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ec420-10ef-40c2-ab39-42b3983114f3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:54.985521 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1378/2753] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Aug 29 23:59:54 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:59:54.990090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09a76f-c610-4c79-939c-cacb9233d3f6 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:55.002302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09a76f-c610-4c79-939c-cacb9233d3f6 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:55.007755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b6c79-d99e-453a-a7a4-a77407f8b5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:55.008349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6c79-d99e-453a-a7a4-a77407f8b5ca admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:55.008621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6c79-d99e-453a-a7a4-a77407f8b5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:55.008890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6c79-d99e-453a-a7a4-a77407f8b5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:55.009534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b6c79-d99e-453a-a7a4-a77407f8b5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:55.011196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b6c79-d99e-453a-a7a4-a77407f8b5ca admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:55.023258 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1376/2754] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:59:55.027478 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ec420-10ef-40c2-ab39-42b3983114f3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:55.038357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ec420-10ef-40c2-ab39-42b3983114f3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:55.043737 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3277a-3bd0-4755-bc21-5525711bf1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:55.044341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3277a-3bd0-4755-bc21-5525711bf1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:55.044614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3277a-3bd0-4755-bc21-5525711bf1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:55.044905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3277a-3bd0-4755-bc21-5525711bf1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:55.045535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3277a-3bd0-4755-bc21-5525711bf1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:55.047256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3277a-3bd0-4755-bc21-5525711bf1b7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:55.057979 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1379/2755] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Aug 29 23:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:59:55.063122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b6c79-d99e-453a-a7a4-a77407f8b5ca admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:55.074008 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b6c79-d99e-453a-a7a4-a77407f8b5ca admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:55.079424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42cbe-eacb-4da6-b8d6-7c547ec54d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:55.080104 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42cbe-eacb-4da6-b8d6-7c547ec54d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:55.080382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42cbe-eacb-4da6-b8d6-7c547ec54d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:55.080669 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42cbe-eacb-4da6-b8d6-7c547ec54d41 admin admin] PATH_INFO: `/v3/projects/36368080bd9247d99ef2e0ecedd08331/users/09547adc7cab4e66b090f71278b22173/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:55.097298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e42cbe-eacb-4da6-b8d6-7c547ec54d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36368080bd9247d99ef2e0ecedd08331, user_id=09547adc7cab4e66b090f71278b22173, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:55.100410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e42cbe-eacb-4da6-b8d6-7c547ec54d41 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:55.184860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1377/2756] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Aug 29 23:59:55 2022] PUT /identity/v3/projects/36368080bd9247d99ef2e0ecedd08331/users/09547adc7cab4e66b090f71278b22173/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:59:55.195832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702a32e-a200-46d9-a4b5-97413a824042 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:55.196119 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702a32e-a200-46d9-a4b5-97413a824042 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:55.196397 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702a32e-a200-46d9-a4b5-97413a824042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:55.225824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6702a32e-a200-46d9-a4b5-97413a824042 None None] MFA Rules not processed for user `09547adc7cab4e66b090f71278b22173`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:59:55.227079 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6702a32e-a200-46d9-a4b5-97413a824042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:55.292540 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1380/2757] 10.222.0.58 () {58 vars in 958 bytes} [Mon Aug 29 23:59:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:59:55.304652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42cbe-eacb-4da6-b8d6-7c547ec54d41 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:59:55.306339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-99e42cbe-eacb-4da6-b8d6-7c547ec54d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:59:55.381107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42cbe-eacb-4da6-b8d6-7c547ec54d41 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:59:55.385009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1f785-6181-44f1-8469-40497f6b477e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:59:55.385608 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1f785-6181-44f1-8469-40497f6b477e service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:59:55.385884 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1f785-6181-44f1-8469-40497f6b477e service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:59:55.386149 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1f785-6181-44f1-8469-40497f6b477e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:59:55.396479 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1f785-6181-44f1-8469-40497f6b477e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:59:55.398120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1f785-6181-44f1-8469-40497f6b477e service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:59:55.408198 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1378/2758] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Aug 29 23:59:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:00:02.637762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702a32e-a200-46d9-a4b5-97413a824042 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:02.687227 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:02.733870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702a32e-a200-46d9-a4b5-97413a824042 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:02.740957 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0674afb-ef4b-406d-8ff3-2787c63fc116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:02.741621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0674afb-ef4b-406d-8ff3-2787c63fc116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:02.741908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0674afb-ef4b-406d-8ff3-2787c63fc116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:02.742168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0674afb-ef4b-406d-8ff3-2787c63fc116 admin admin] PATH_INFO: `/v3/users/09547adc7cab4e66b090f71278b22173` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:02.743425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0674afb-ef4b-406d-8ff3-2787c63fc116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09547adc7cab4e66b090f71278b22173)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:02.745451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0674afb-ef4b-406d-8ff3-2787c63fc116 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:02.995457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d0674afb-ef4b-406d-8ff3-2787c63fc116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09547adc7cab4e66b090f71278b22173'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:03.019918 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d0674afb-ef4b-406d-8ff3-2787c63fc116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09547adc7cab4e66b090f71278b22173'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:03.041614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d0674afb-ef4b-406d-8ff3-2787c63fc116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09547adc7cab4e66b090f71278b22173'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:03.124300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1f785-6181-44f1-8469-40497f6b477e service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:03.129974 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d3b1f785-6181-44f1-8469-40497f6b477e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:03.141443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1f785-6181-44f1-8469-40497f6b477e service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:03.149683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d0674afb-ef4b-406d-8ff3-2787c63fc116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09547adc7cab4e66b090f71278b22173'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:03.150568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d0674afb-ef4b-406d-8ff3-2787c63fc116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09547adc7cab4e66b090f71278b22173'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:03.151461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:03.153075 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:03.153551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:03.153985 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] PATH_INFO: `/v3/users/803816a2b15d456b981bcc397a6aac17` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:03.156028 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=803816a2b15d456b981bcc397a6aac17)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:03.159685 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:03.160983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1381/2759] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:02 2022] DELETE /identity/v3/users/09547adc7cab4e66b090f71278b22173 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:03.166542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e17285-8e18-4661-aba3-09f720818bc6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:03.186401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e17285-8e18-4661-aba3-09f720818bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:03.186401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e17285-8e18-4661-aba3-09f720818bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:03.188617 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:03.193054 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a6e17285-8e18-4661-aba3-09f720818bc6 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:03.194503 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a6e17285-8e18-4661-aba3-09f720818bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:03.284281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1382/2760] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:00:03.295497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e17285-8e18-4661-aba3-09f720818bc6 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:03.297025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a6e17285-8e18-4661-aba3-09f720818bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:03.374669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:03.388090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e17285-8e18-4661-aba3-09f720818bc6 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:03.392290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85393f0-dc2c-45a1-ab2b-46b83883c008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:03.392933 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85393f0-dc2c-45a1-ab2b-46b83883c008 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:03.393206 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85393f0-dc2c-45a1-ab2b-46b83883c008 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:03.393479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85393f0-dc2c-45a1-ab2b-46b83883c008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:03.404207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85393f0-dc2c-45a1-ab2b-46b83883c008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:03.405696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85393f0-dc2c-45a1-ab2b-46b83883c008 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:03.416586 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1383/2761] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:00:03.455461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '803816a2b15d456b981bcc397a6aac17'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:03.473761 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '803816a2b15d456b981bcc397a6aac17'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:03.485599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '803816a2b15d456b981bcc397a6aac17'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:03.585334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '803816a2b15d456b981bcc397a6aac17'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:03.586380 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '803816a2b15d456b981bcc397a6aac17'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:03.595181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1379/2762] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:03 2022] DELETE /identity/v3/users/803816a2b15d456b981bcc397a6aac17 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:00:03.599122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3afa2a-bcaf-48eb-b930-77fbbc2c4232 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:03.599402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3afa2a-bcaf-48eb-b930-77fbbc2c4232 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:03.599675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3afa2a-bcaf-48eb-b930-77fbbc2c4232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:03.620203 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4f3afa2a-bcaf-48eb-b930-77fbbc2c4232 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:03.622033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4f3afa2a-bcaf-48eb-b930-77fbbc2c4232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:03.692204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1384/2763] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:00:03.703215 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:03.705183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:03.799114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbc88a-6bfd-4570-9af6-0859c7413f48 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:03.804276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0983a5-27e8-4c44-a153-f981475b2c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:03.804965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0983a5-27e8-4c44-a153-f981475b2c9a service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:03.805269 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0983a5-27e8-4c44-a153-f981475b2c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:03.805544 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0983a5-27e8-4c44-a153-f981475b2c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:03.818300 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0983a5-27e8-4c44-a153-f981475b2c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:03.820284 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0983a5-27e8-4c44-a153-f981475b2c9a service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:03.833232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1380/2764] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:00:04.070340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3afa2a-bcaf-48eb-b930-77fbbc2c4232 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:04.072232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4f3afa2a-bcaf-48eb-b930-77fbbc2c4232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:04.087066 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3afa2a-bcaf-48eb-b930-77fbbc2c4232 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:04.093671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:04.094335 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:04.094711 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:04.095086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] PATH_INFO: `/v3/projects/36368080bd9247d99ef2e0ecedd08331` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:04.113258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=36368080bd9247d99ef2e0ecedd08331)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:04.115405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:04.286295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36368080bd9247d99ef2e0ecedd08331', 'request_id': 'req-de6abbdb-78b2-4b16-bc67-507eb3226e2e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:04.298699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36368080bd9247d99ef2e0ecedd08331', 'request_id': 'req-de6abbdb-78b2-4b16-bc67-507eb3226e2e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:04.402029 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36368080bd9247d99ef2e0ecedd08331', 'request_id': 'req-de6abbdb-78b2-4b16-bc67-507eb3226e2e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:04.417719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] The token cache is being invalidate because project 36368080bd9247d99ef2e0ecedd08331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:04.417719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:04.422255 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:04.424869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1385/2765] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:04 2022] DELETE /identity/v3/projects/36368080bd9247d99ef2e0ecedd08331 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:04.430033 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23249-d32a-4979-bcbd-77e093dbaa34 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:04.441266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23249-d32a-4979-bcbd-77e093dbaa34 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:04.441266 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23249-d32a-4979-bcbd-77e093dbaa34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:04.453858 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:04.456283 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:04.458477 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-d5f23249-d32a-4979-bcbd-77e093dbaa34 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:04.459782 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d5f23249-d32a-4979-bcbd-77e093dbaa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:04.546286 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1381/2766] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:04.550406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6abbdb-78b2-4b16-bc67-507eb3226e2e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:04.555701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:04.556585 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:04.556871 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:04.557144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] PATH_INFO: `/v3/users/44a591ea1e4846b2b248b00f808d27f0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:04.558520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=44a591ea1e4846b2b248b00f808d27f0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:04.561090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:04.585085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:04.618415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08cdc8-4633-4041-9245-77c1c0c84c42 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:04.618415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08cdc8-4633-4041-9245-77c1c0c84c42 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:04.618415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08cdc8-4633-4041-9245-77c1c0c84c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:04.642750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-1c08cdc8-4633-4041-9245-77c1c0c84c42 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:04.644001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1c08cdc8-4633-4041-9245-77c1c0c84c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:04.654291 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1382/2767] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:00:04.658130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08cdc8-4633-4041-9245-77c1c0c84c42 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:04.658130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-1c08cdc8-4633-4041-9245-77c1c0c84c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:04.671865 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08cdc8-4633-4041-9245-77c1c0c84c42 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:04.678563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:04.679350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:04.679730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:04.680107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] PATH_INFO: `/v3/projects/6f71249706794faeb24cd0708d6d0698` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:04.681460 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f71249706794faeb24cd0708d6d0698)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:04.683576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:04.807200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44a591ea1e4846b2b248b00f808d27f0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:04.821119 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44a591ea1e4846b2b248b00f808d27f0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:04.834502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44a591ea1e4846b2b248b00f808d27f0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:04.835494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:04.852254 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f71249706794faeb24cd0708d6d0698', 'request_id': 'req-d0964ae2-3ab2-40ec-ba15-5c349a781603'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:04.975358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44a591ea1e4846b2b248b00f808d27f0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:04.976016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44a591ea1e4846b2b248b00f808d27f0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:04.984620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1386/2768] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:04 2022] DELETE /identity/v3/users/44a591ea1e4846b2b248b00f808d27f0 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:04.985800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:04.987671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:05.053785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f71249706794faeb24cd0708d6d0698', 'request_id': 'req-d0964ae2-3ab2-40ec-ba15-5c349a781603'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:05.061431 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f71249706794faeb24cd0708d6d0698', 'request_id': 'req-d0964ae2-3ab2-40ec-ba15-5c349a781603'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:05.065524 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480deb8-1693-46eb-8b1d-4f1a3feb9e3f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:05.070542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2a726-2341-4fc4-b941-56b199f06b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:05.071164 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2a726-2341-4fc4-b941-56b199f06b6c admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:05.071448 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2a726-2341-4fc4-b941-56b199f06b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:05.071708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2a726-2341-4fc4-b941-56b199f06b6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:05.072373 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2a726-2341-4fc4-b941-56b199f06b6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:00:05.072684 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2a726-2341-4fc4-b941-56b199f06b6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:05.075074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2a726-2341-4fc4-b941-56b199f06b6c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:05.089268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] The token cache is being invalidate because project 6f71249706794faeb24cd0708d6d0698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:05.089268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:05.089268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-d0964ae2-3ab2-40ec-ba15-5c349a781603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:05.089268 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1383/2769] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:04 2022] DELETE /identity/v3/projects/6f71249706794faeb24cd0708d6d0698 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:05.089837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1387/2770] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 30 00:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:00:05.090941 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e93b8-d70e-4dd5-be02-55f6aaf373ce None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:05.091231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e93b8-d70e-4dd5-be02-55f6aaf373ce None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:05.091506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e93b8-d70e-4dd5-be02-55f6aaf373ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:05.091989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafdec7c-8436-4424-bb89-0293c9e68223 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:05.092576 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafdec7c-8436-4424-bb89-0293c9e68223 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:05.093038 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafdec7c-8436-4424-bb89-0293c9e68223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:05.113726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-aafdec7c-8436-4424-bb89-0293c9e68223 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:05.115047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-aafdec7c-8436-4424-bb89-0293c9e68223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:05.122163 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8a0e93b8-d70e-4dd5-be02-55f6aaf373ce None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:05.123986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1384/2771] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:05.124210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8a0e93b8-d70e-4dd5-be02-55f6aaf373ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:05.124881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafdec7c-8436-4424-bb89-0293c9e68223 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:05.126206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-aafdec7c-8436-4424-bb89-0293c9e68223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:05.135571 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1388/2772] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:05.136428 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e93b8-d70e-4dd5-be02-55f6aaf373ce None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:05.138914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8a0e93b8-d70e-4dd5-be02-55f6aaf373ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:05.156383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafdec7c-8436-4424-bb89-0293c9e68223 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:05.161681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a582b56-2720-4332-b444-5ad0169447ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:05.162446 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a582b56-2720-4332-b444-5ad0169447ce admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:05.162722 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a582b56-2720-4332-b444-5ad0169447ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:05.162992 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a582b56-2720-4332-b444-5ad0169447ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:05.163649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a582b56-2720-4332-b444-5ad0169447ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:05.165762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a582b56-2720-4332-b444-5ad0169447ce admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:05.208182 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e93b8-d70e-4dd5-be02-55f6aaf373ce None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:05.212111 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce84c63-d22d-4e1d-99c7-6c923a733a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:05.212729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce84c63-d22d-4e1d-99c7-6c923a733a42 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:05.213007 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce84c63-d22d-4e1d-99c7-6c923a733a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:05.213279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce84c63-d22d-4e1d-99c7-6c923a733a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:05.224251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce84c63-d22d-4e1d-99c7-6c923a733a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:05.225743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce84c63-d22d-4e1d-99c7-6c923a733a42 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:05.237026 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1389/2773] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:00:05.238319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be950af6-48b4-46e8-acdc-abb2553d53df None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:05.238633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be950af6-48b4-46e8-acdc-abb2553d53df None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:05.247124 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be950af6-48b4-46e8-acdc-abb2553d53df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:05.260969 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-be950af6-48b4-46e8-acdc-abb2553d53df None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:05.262975 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-be950af6-48b4-46e8-acdc-abb2553d53df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:05.272005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1390/2774] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:05.276425 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be950af6-48b4-46e8-acdc-abb2553d53df None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:05.286505 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be950af6-48b4-46e8-acdc-abb2553d53df None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:05.291598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6b6b-2127-4885-918a-63d29305c695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:05.292248 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6b6b-2127-4885-918a-63d29305c695 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:05.292527 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6b6b-2127-4885-918a-63d29305c695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:05.292802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6b6b-2127-4885-918a-63d29305c695 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:05.294065 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec6b6b-2127-4885-918a-63d29305c695 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:00:05.294376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec6b6b-2127-4885-918a-63d29305c695 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:05.296094 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec6b6b-2127-4885-918a-63d29305c695 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:05.308752 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1391/2775] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 30 00:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:00:05.313584 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6b6b-2127-4885-918a-63d29305c695 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:05.319023 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:05.333023 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6b6b-2127-4885-918a-63d29305c695 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:05.333313 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1385/2776] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 30 00:00:05 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:05.338110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a582b56-2720-4332-b444-5ad0169447ce admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:05.412904 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863a765-50e6-49ce-a573-6c5348965233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:05.413710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863a765-50e6-49ce-a573-6c5348965233 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:05.414280 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863a765-50e6-49ce-a573-6c5348965233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:05.447763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863a765-50e6-49ce-a573-6c5348965233 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:05.447763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863a765-50e6-49ce-a573-6c5348965233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:05.447763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863a765-50e6-49ce-a573-6c5348965233 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:05.448298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a582b56-2720-4332-b444-5ad0169447ce admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:05.448298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895dbc1-bcd2-4474-bae3-956e5f7281f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:05.448298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895dbc1-bcd2-4474-bae3-956e5f7281f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:05.448298 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895dbc1-bcd2-4474-bae3-956e5f7281f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:05.450097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895dbc1-bcd2-4474-bae3-956e5f7281f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:05.450097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895dbc1-bcd2-4474-bae3-956e5f7281f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:05.450097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895dbc1-bcd2-4474-bae3-956e5f7281f0 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:06.316760 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1386/2777] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 30 00:00:05 2022] POST /identity/v3/users => generated 291 bytes in 979 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:00:06.320988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1392/2778] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 30 00:00:05 2022] POST /identity/v3/projects => generated 373 bytes in 1005 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:00:06.321525 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895dbc1-bcd2-4474-bae3-956e5f7281f0 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:06.323058 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863a765-50e6-49ce-a573-6c5348965233 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:06.336863 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895dbc1-bcd2-4474-bae3-956e5f7281f0 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:06.401142 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863a765-50e6-49ce-a573-6c5348965233 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:06.407243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f5da4-2de9-4e5c-aaf5-78f77a0892e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:06.414509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f5da4-2de9-4e5c-aaf5-78f77a0892e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:06.414509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f5da4-2de9-4e5c-aaf5-78f77a0892e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:06.414509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f5da4-2de9-4e5c-aaf5-78f77a0892e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:06.414509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f5da4-2de9-4e5c-aaf5-78f77a0892e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:06.414509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f5da4-2de9-4e5c-aaf5-78f77a0892e5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:06.416972 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cead57-404a-490b-83f1-726ce6ffe8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:06.417801 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cead57-404a-490b-83f1-726ce6ffe8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:06.418230 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cead57-404a-490b-83f1-726ce6ffe8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:06.418608 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cead57-404a-490b-83f1-726ce6ffe8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:06.419353 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cead57-404a-490b-83f1-726ce6ffe8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:06.421371 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cead57-404a-490b-83f1-726ce6ffe8c0 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:06.433643 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1387/2779] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:00:06.438110 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cead57-404a-490b-83f1-726ce6ffe8c0 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:06.448846 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cead57-404a-490b-83f1-726ce6ffe8c0 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:06.454703 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e484dd-f666-431a-b52c-a1d77b672a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:06.455334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e484dd-f666-431a-b52c-a1d77b672a69 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:06.455614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e484dd-f666-431a-b52c-a1d77b672a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:06.455886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e484dd-f666-431a-b52c-a1d77b672a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:06.456604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e484dd-f666-431a-b52c-a1d77b672a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:06.458220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e484dd-f666-431a-b52c-a1d77b672a69 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:06.478356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1388/2780] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:00:06.479982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e484dd-f666-431a-b52c-a1d77b672a69 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:06.483236 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-12e484dd-f666-431a-b52c-a1d77b672a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:06.496055 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:06.506939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e484dd-f666-431a-b52c-a1d77b672a69 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:06.514338 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:06.515363 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:06.515575 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:06.515794 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] PATH_INFO: `/v3/projects/b86fe5a13c7e41c787fe6e8ab44d3b73` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:06.518679 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b86fe5a13c7e41c787fe6e8ab44d3b73)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:06.521307 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:06.569915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1393/2781] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 30 00:00:06 2022] POST /identity/v3/users => generated 287 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:06.571380 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f5da4-2de9-4e5c-aaf5-78f77a0892e5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:06.587920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f5da4-2de9-4e5c-aaf5-78f77a0892e5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:06.594642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e90d2a-a657-44da-bca3-180c469f95b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:06.595682 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e90d2a-a657-44da-bca3-180c469f95b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:06.596090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e90d2a-a657-44da-bca3-180c469f95b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:06.596465 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e90d2a-a657-44da-bca3-180c469f95b6 admin admin] PATH_INFO: `/v3/projects/3771a66c80964a40805c3a9f1f51c3da/users/bf8fecbb57394e2e8f5dabfd75ac6b7c/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:06.613661 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e90d2a-a657-44da-bca3-180c469f95b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3771a66c80964a40805c3a9f1f51c3da, user_id=bf8fecbb57394e2e8f5dabfd75ac6b7c, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:06.616701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e90d2a-a657-44da-bca3-180c469f95b6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:06.717070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1394/2782] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:06 2022] PUT /identity/v3/projects/3771a66c80964a40805c3a9f1f51c3da/users/bf8fecbb57394e2e8f5dabfd75ac6b7c/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:06.722992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e90d2a-a657-44da-bca3-180c469f95b6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:06.733381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b86fe5a13c7e41c787fe6e8ab44d3b73', 'request_id': 'req-14984239-8b47-474c-96ab-9c307128bee0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:06.808344 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b86fe5a13c7e41c787fe6e8ab44d3b73', 'request_id': 'req-14984239-8b47-474c-96ab-9c307128bee0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:06.827892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b86fe5a13c7e41c787fe6e8ab44d3b73', 'request_id': 'req-14984239-8b47-474c-96ab-9c307128bee0'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:06.837944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:06.845699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] The token cache is being invalidate because project b86fe5a13c7e41c787fe6e8ab44d3b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:06.846292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:06.847490 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-14984239-8b47-474c-96ab-9c307128bee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:06.850275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1389/2783] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:06 2022] DELETE /identity/v3/projects/b86fe5a13c7e41c787fe6e8ab44d3b73 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:06.851415 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e90d2a-a657-44da-bca3-180c469f95b6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:06.852548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fbf85-a5c9-4aa6-bd64-ff683b192d26 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:06.853045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fbf85-a5c9-4aa6-bd64-ff683b192d26 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:06.854370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fbf85-a5c9-4aa6-bd64-ff683b192d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:06.883481 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-525fbf85-a5c9-4aa6-bd64-ff683b192d26 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:06.884724 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-525fbf85-a5c9-4aa6-bd64-ff683b192d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:06.948365 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936763a-0bb6-4548-be05-881a6a9bdba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:06.949085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936763a-0bb6-4548-be05-881a6a9bdba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:06.949922 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936763a-0bb6-4548-be05-881a6a9bdba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:06.950201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936763a-0bb6-4548-be05-881a6a9bdba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:06.950897 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936763a-0bb6-4548-be05-881a6a9bdba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:06.952531 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936763a-0bb6-4548-be05-881a6a9bdba4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:06.963381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1395/2784] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:00:06.963381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936763a-0bb6-4548-be05-881a6a9bdba4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:06.963381 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3936763a-0bb6-4548-be05-881a6a9bdba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:06.968225 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1390/2785] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:06.970254 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525fbf85-a5c9-4aa6-bd64-ff683b192d26 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:06.982834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-525fbf85-a5c9-4aa6-bd64-ff683b192d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:06.983912 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936763a-0bb6-4548-be05-881a6a9bdba4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:06.988889 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:06.989645 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:06.989920 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:06.990205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] PATH_INFO: `/v3/users/4b5f416ac07f40d9aad0438385496899` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:06.991555 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b5f416ac07f40d9aad0438385496899)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:06.992963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525fbf85-a5c9-4aa6-bd64-ff683b192d26 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:06.993542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:06.999160 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a156741-7d25-4596-a0cd-5a93e152b4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:07.000023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a156741-7d25-4596-a0cd-5a93e152b4da admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:07.008506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a156741-7d25-4596-a0cd-5a93e152b4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:07.008506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a156741-7d25-4596-a0cd-5a93e152b4da admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:07.008506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a156741-7d25-4596-a0cd-5a93e152b4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:07.008506 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a156741-7d25-4596-a0cd-5a93e152b4da admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:07.021951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1391/2786] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:00:07.021951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a156741-7d25-4596-a0cd-5a93e152b4da admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:07.021951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9a156741-7d25-4596-a0cd-5a93e152b4da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:07.097718 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a156741-7d25-4596-a0cd-5a93e152b4da admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:07.103803 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bae3d-dd6b-4c90-afbb-7f9ad44a7612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:07.104786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bae3d-dd6b-4c90-afbb-7f9ad44a7612 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:07.105139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bae3d-dd6b-4c90-afbb-7f9ad44a7612 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:07.105448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bae3d-dd6b-4c90-afbb-7f9ad44a7612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:07.118008 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bae3d-dd6b-4c90-afbb-7f9ad44a7612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:07.120278 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bae3d-dd6b-4c90-afbb-7f9ad44a7612 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:07.132905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1392/2787] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:00:07.134111 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bae3d-dd6b-4c90-afbb-7f9ad44a7612 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:07.145324 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bae3d-dd6b-4c90-afbb-7f9ad44a7612 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:07.150147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7a280-45ad-4062-9f66-9c74d5720d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:07.150853 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7a280-45ad-4062-9f66-9c74d5720d55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:07.151255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7a280-45ad-4062-9f66-9c74d5720d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:07.151468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7a280-45ad-4062-9f66-9c74d5720d55 admin admin] PATH_INFO: `/v3/projects/afc6be07438a47fb9b13974bd61fbe57/users/42a2a81700ff44c394602c9a5fac5df0/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:07.170372 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7a280-45ad-4062-9f66-9c74d5720d55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc6be07438a47fb9b13974bd61fbe57, user_id=42a2a81700ff44c394602c9a5fac5df0, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:07.173218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7a280-45ad-4062-9f66-9c74d5720d55 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:07.203247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b5f416ac07f40d9aad0438385496899'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:07.215951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b5f416ac07f40d9aad0438385496899'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:07.223504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b5f416ac07f40d9aad0438385496899'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:07.300049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b5f416ac07f40d9aad0438385496899'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:07.301014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b5f416ac07f40d9aad0438385496899'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:07.319059 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1393/2788] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:07 2022] PUT /identity/v3/projects/afc6be07438a47fb9b13974bd61fbe57/users/42a2a81700ff44c394602c9a5fac5df0/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:07.319771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1396/2789] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:06 2022] DELETE /identity/v3/users/4b5f416ac07f40d9aad0438385496899 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:07.332521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13275823-12c5-49f3-a829-692d4e372c02 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:07.332813 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13275823-12c5-49f3-a829-692d4e372c02 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:07.333018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13275823-12c5-49f3-a829-692d4e372c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:07.375014 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-13275823-12c5-49f3-a829-692d4e372c02 None None] MFA Rules not processed for user `42a2a81700ff44c394602c9a5fac5df0`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:07.376332 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-13275823-12c5-49f3-a829-692d4e372c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:07.425614 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:07.463257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1394/2790] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:07.482341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:07.484541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:07.548137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53f091-0277-4e1e-9968-ee4f354cca00 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:07.552127 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087df900-d43c-43ff-ba84-049f7ef401ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:07.552959 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087df900-d43c-43ff-ba84-049f7ef401ac service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:07.553232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087df900-d43c-43ff-ba84-049f7ef401ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:07.553498 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087df900-d43c-43ff-ba84-049f7ef401ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:07.563744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087df900-d43c-43ff-ba84-049f7ef401ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:07.565265 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087df900-d43c-43ff-ba84-049f7ef401ac service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:07.575879 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1397/2791] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:00:07.870218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13275823-12c5-49f3-a829-692d4e372c02 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:07.872461 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-13275823-12c5-49f3-a829-692d4e372c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:07.990626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13275823-12c5-49f3-a829-692d4e372c02 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:07.995487 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:07.997155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:07.997155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:07.997538 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] PATH_INFO: `/v3/projects/38db15f31dca445e8defb77b3f0c5770` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:07.998912 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=38db15f31dca445e8defb77b3f0c5770)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:08.000941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:08.153089 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38db15f31dca445e8defb77b3f0c5770', 'request_id': 'req-2bacbfc0-747e-4157-8b4a-e4caf97686ea'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:08.235358 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38db15f31dca445e8defb77b3f0c5770', 'request_id': 'req-2bacbfc0-747e-4157-8b4a-e4caf97686ea'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:08.241963 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38db15f31dca445e8defb77b3f0c5770', 'request_id': 'req-2bacbfc0-747e-4157-8b4a-e4caf97686ea'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:08.257270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] The token cache is being invalidate because project 38db15f31dca445e8defb77b3f0c5770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:08.257270 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:08.257673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2bacbfc0-747e-4157-8b4a-e4caf97686ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:08.259467 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1395/2792] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:07 2022] DELETE /identity/v3/projects/38db15f31dca445e8defb77b3f0c5770 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:08.264536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebf8d8-e533-4c44-959f-c4fa3e7b25f6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:08.265093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebf8d8-e533-4c44-959f-c4fa3e7b25f6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:08.265554 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebf8d8-e533-4c44-959f-c4fa3e7b25f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:08.288192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-beebf8d8-e533-4c44-959f-c4fa3e7b25f6 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:08.289455 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-beebf8d8-e533-4c44-959f-c4fa3e7b25f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:08.295796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:08.357792 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1398/2793] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:08.365789 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2d15-37f6-4e75-a364-22000611200b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:08.366068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2d15-37f6-4e75-a364-22000611200b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:08.366508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2d15-37f6-4e75-a364-22000611200b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:08.393093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-516e2d15-37f6-4e75-a364-22000611200b None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:08.393093 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-516e2d15-37f6-4e75-a364-22000611200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:08.397899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1396/2794] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:08.403070 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebf8d8-e533-4c44-959f-c4fa3e7b25f6 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:08.414231 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebf8d8-e533-4c44-959f-c4fa3e7b25f6 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:08.420494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e7cac-e3a1-42ef-91e9-949a8ce718cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:08.421289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7cac-e3a1-42ef-91e9-949a8ce718cd admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:08.421650 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7cac-e3a1-42ef-91e9-949a8ce718cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:08.422080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7cac-e3a1-42ef-91e9-949a8ce718cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:08.423051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e7cac-e3a1-42ef-91e9-949a8ce718cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:00:08.423581 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e7cac-e3a1-42ef-91e9-949a8ce718cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:08.426215 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e7cac-e3a1-42ef-91e9-949a8ce718cd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:08.444295 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1399/2795] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 30 00:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:00:08.450090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e2d15-37f6-4e75-a364-22000611200b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:08.469256 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e2d15-37f6-4e75-a364-22000611200b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:08.479959 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816b165-7216-4653-a86b-9efeee18f6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:08.481804 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816b165-7216-4653-a86b-9efeee18f6bf admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:08.482315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816b165-7216-4653-a86b-9efeee18f6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:08.482879 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816b165-7216-4653-a86b-9efeee18f6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:08.484090 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816b165-7216-4653-a86b-9efeee18f6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:08.487965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816b165-7216-4653-a86b-9efeee18f6bf admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:08.497890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:08.643830 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1397/2796] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 30 00:00:08 2022] POST /identity/v3/projects => generated 371 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:08.648310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e7cac-e3a1-42ef-91e9-949a8ce718cd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:08.663859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e7cac-e3a1-42ef-91e9-949a8ce718cd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:08.695145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501847f-1304-4fcc-914d-a5881047d466 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:08.695337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501847f-1304-4fcc-914d-a5881047d466 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:08.695547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501847f-1304-4fcc-914d-a5881047d466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:08.729420 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-3501847f-1304-4fcc-914d-a5881047d466 None None] MFA Rules not processed for user `bf8fecbb57394e2e8f5dabfd75ac6b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:08.731987 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3501847f-1304-4fcc-914d-a5881047d466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:08.743836 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93fa0c-29ca-4c76-9ba9-b54848f8d56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:08.744642 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93fa0c-29ca-4c76-9ba9-b54848f8d56a admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:08.744951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93fa0c-29ca-4c76-9ba9-b54848f8d56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:08.747122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93fa0c-29ca-4c76-9ba9-b54848f8d56a admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:08.747990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93fa0c-29ca-4c76-9ba9-b54848f8d56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:08.752139 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93fa0c-29ca-4c76-9ba9-b54848f8d56a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:08.822082 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1398/2797] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:08.836081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501847f-1304-4fcc-914d-a5881047d466 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:08.837520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3501847f-1304-4fcc-914d-a5881047d466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:08.900096 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1400/2798] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 30 00:00:08 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:08.900775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501847f-1304-4fcc-914d-a5881047d466 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:08.904816 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93fa0c-29ca-4c76-9ba9-b54848f8d56a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:08.905341 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75867d3e-0dc2-4747-88c1-b7a26866366d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:08.905936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867d3e-0dc2-4747-88c1-b7a26866366d service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:08.906227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867d3e-0dc2-4747-88c1-b7a26866366d service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:08.906532 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867d3e-0dc2-4747-88c1-b7a26866366d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:08.916264 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93fa0c-29ca-4c76-9ba9-b54848f8d56a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:08.916819 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75867d3e-0dc2-4747-88c1-b7a26866366d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:08.918856 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75867d3e-0dc2-4747-88c1-b7a26866366d service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:08.922843 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f938f69-e74a-4a95-bb85-c4754412758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:08.923077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f938f69-e74a-4a95-bb85-c4754412758c admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:08.923211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f938f69-e74a-4a95-bb85-c4754412758c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:08.923485 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f938f69-e74a-4a95-bb85-c4754412758c admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:08.924132 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f938f69-e74a-4a95-bb85-c4754412758c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:08.925800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f938f69-e74a-4a95-bb85-c4754412758c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:08.929932 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1399/2799] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:00:08.937129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1401/2800] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:00:08.946757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75867d3e-0dc2-4747-88c1-b7a26866366d service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:08.958337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75867d3e-0dc2-4747-88c1-b7a26866366d service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:08.963326 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e4451-f53c-4440-a1c8-c053507c55f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:08.963931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e4451-f53c-4440-a1c8-c053507c55f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:08.964216 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e4451-f53c-4440-a1c8-c053507c55f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:08.964489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e4451-f53c-4440-a1c8-c053507c55f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:08.965731 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e4451-f53c-4440-a1c8-c053507c55f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:08.967370 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e4451-f53c-4440-a1c8-c053507c55f4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:08.979211 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1400/2801] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:00:08.984394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f938f69-e74a-4a95-bb85-c4754412758c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:08.994659 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f938f69-e74a-4a95-bb85-c4754412758c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:09.000042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f5ee5-ac8c-4e26-9f33-c44246d6578d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:09.000852 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f5ee5-ac8c-4e26-9f33-c44246d6578d admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:09.001130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f5ee5-ac8c-4e26-9f33-c44246d6578d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:09.001405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f5ee5-ac8c-4e26-9f33-c44246d6578d admin admin] PATH_INFO: `/v3/projects/29197913955b4bdca450928038af981d/users/82bafb24a014425984618c745e7e0e90/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:09.019554 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f5ee5-ac8c-4e26-9f33-c44246d6578d admin admin] RBAC: Authorizing `identity:create_grant(project_id=29197913955b4bdca450928038af981d, user_id=82bafb24a014425984618c745e7e0e90, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:09.022211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f5ee5-ac8c-4e26-9f33-c44246d6578d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:09.112082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1402/2802] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:08 2022] PUT /identity/v3/projects/29197913955b4bdca450928038af981d/users/82bafb24a014425984618c745e7e0e90/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:09.127868 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4be1a-7c1f-440c-bbd9-908c71cf50c7 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:09.128199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4be1a-7c1f-440c-bbd9-908c71cf50c7 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:09.128516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4be1a-7c1f-440c-bbd9-908c71cf50c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:09.165905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7ae4be1a-7c1f-440c-bbd9-908c71cf50c7 None None] MFA Rules not processed for user `82bafb24a014425984618c745e7e0e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:09.167792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7ae4be1a-7c1f-440c-bbd9-908c71cf50c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:09.243666 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1401/2803] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:09.248496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f5ee5-ac8c-4e26-9f33-c44246d6578d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:09.311908 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f5ee5-ac8c-4e26-9f33-c44246d6578d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:09.321630 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d7496-bab7-4a7b-b80a-0952771ccdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:09.322442 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d7496-bab7-4a7b-b80a-0952771ccdba admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:09.322839 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d7496-bab7-4a7b-b80a-0952771ccdba admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:09.323207 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d7496-bab7-4a7b-b80a-0952771ccdba admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:09.325250 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d7496-bab7-4a7b-b80a-0952771ccdba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:09.326564 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d7496-bab7-4a7b-b80a-0952771ccdba admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:09.456053 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1403/2804] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 30 00:00:09 2022] POST /identity/v3/projects => generated 373 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:09.461588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4be1a-7c1f-440c-bbd9-908c71cf50c7 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:09.475282 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4be1a-7c1f-440c-bbd9-908c71cf50c7 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:09.543874 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b81d25-470d-4e17-af6b-d2a2fd4801b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:09.544498 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81d25-470d-4e17-af6b-d2a2fd4801b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:09.544776 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81d25-470d-4e17-af6b-d2a2fd4801b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:09.545044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81d25-470d-4e17-af6b-d2a2fd4801b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:09.545699 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b81d25-470d-4e17-af6b-d2a2fd4801b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:09.547674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b81d25-470d-4e17-af6b-d2a2fd4801b8 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:09.595708 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba092a-0e3d-4e96-a8bd-29f9c5c15af9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:09.596309 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba092a-0e3d-4e96-a8bd-29f9c5c15af9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:09.596755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba092a-0e3d-4e96-a8bd-29f9c5c15af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:09.598223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:09.618598 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9dba092a-0e3d-4e96-a8bd-29f9c5c15af9 None None] MFA Rules not processed for user `42a2a81700ff44c394602c9a5fac5df0`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:09.619836 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9dba092a-0e3d-4e96-a8bd-29f9c5c15af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:09.669445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1402/2805] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 30 00:00:09 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:00:09.670676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b81d25-470d-4e17-af6b-d2a2fd4801b8 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:09.679615 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1404/2806] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:00:09.680356 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b81d25-470d-4e17-af6b-d2a2fd4801b8 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:09.686012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbf115-ba46-4efb-92ec-3733aa44a302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:09.686801 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf115-ba46-4efb-92ec-3733aa44a302 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:09.686975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf115-ba46-4efb-92ec-3733aa44a302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:09.687233 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf115-ba46-4efb-92ec-3733aa44a302 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:09.687890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbf115-ba46-4efb-92ec-3733aa44a302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:09.689489 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbf115-ba46-4efb-92ec-3733aa44a302 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:09.701926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1403/2807] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:00:09.706326 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba092a-0e3d-4e96-a8bd-29f9c5c15af9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:09.727399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba092a-0e3d-4e96-a8bd-29f9c5c15af9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:09.734769 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c1d8a-8de8-4a9f-9abe-a87fe0a66a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:09.735713 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c1d8a-8de8-4a9f-9abe-a87fe0a66a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:09.736082 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c1d8a-8de8-4a9f-9abe-a87fe0a66a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:09.736433 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c1d8a-8de8-4a9f-9abe-a87fe0a66a40 admin admin] PATH_INFO: `/v3/projects/dd70dd977b154b65976049bfb8ba24bf/users/8ac914bb88044a3a97e1b2d56fdd35d0/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:09.755628 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c1d8a-8de8-4a9f-9abe-a87fe0a66a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd70dd977b154b65976049bfb8ba24bf, user_id=8ac914bb88044a3a97e1b2d56fdd35d0, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:09.758626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c1d8a-8de8-4a9f-9abe-a87fe0a66a40 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:09.865310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:09.928800 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1405/2808] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:09 2022] PUT /identity/v3/projects/dd70dd977b154b65976049bfb8ba24bf/users/8ac914bb88044a3a97e1b2d56fdd35d0/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:00:09.932934 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbf115-ba46-4efb-92ec-3733aa44a302 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:09.943535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbf115-ba46-4efb-92ec-3733aa44a302 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:10.009860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3bee2-244d-4ded-b694-5048fcf1ef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:10.010500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3bee2-244d-4ded-b694-5048fcf1ef05 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:10.010786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3bee2-244d-4ded-b694-5048fcf1ef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:10.011063 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3bee2-244d-4ded-b694-5048fcf1ef05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:10.012131 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3bee2-244d-4ded-b694-5048fcf1ef05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:10.013457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3bee2-244d-4ded-b694-5048fcf1ef05 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:10.028405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1404/2809] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:00:10.033633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c1d8a-8de8-4a9f-9abe-a87fe0a66a40 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:10.045766 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c1d8a-8de8-4a9f-9abe-a87fe0a66a40 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:10.050663 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29abb5-8b1e-4d47-96ce-4e49901dad1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:10.051342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29abb5-8b1e-4d47-96ce-4e49901dad1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:10.051823 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29abb5-8b1e-4d47-96ce-4e49901dad1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:10.052104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29abb5-8b1e-4d47-96ce-4e49901dad1b admin admin] PATH_INFO: `/v3/projects/dd70dd977b154b65976049bfb8ba24bf/users/8ac914bb88044a3a97e1b2d56fdd35d0/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:10.054086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29abb5-8b1e-4d47-96ce-4e49901dad1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd70dd977b154b65976049bfb8ba24bf, user_id=8ac914bb88044a3a97e1b2d56fdd35d0, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:10.056796 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29abb5-8b1e-4d47-96ce-4e49901dad1b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:10.068958 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1406/2810] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:10 2022] PUT /identity/v3/projects/dd70dd977b154b65976049bfb8ba24bf/users/8ac914bb88044a3a97e1b2d56fdd35d0/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:00:10.080500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc140a-db75-466b-89df-06b1a7c8c413 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:10.080790 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc140a-db75-466b-89df-06b1a7c8c413 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:10.081073 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc140a-db75-466b-89df-06b1a7c8c413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:10.112361 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-61dc140a-db75-466b-89df-06b1a7c8c413 None None] MFA Rules not processed for user `8ac914bb88044a3a97e1b2d56fdd35d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:10.113590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-61dc140a-db75-466b-89df-06b1a7c8c413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:10.189339 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1405/2811] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:00:10.202672 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29abb5-8b1e-4d47-96ce-4e49901dad1b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:10.267748 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29abb5-8b1e-4d47-96ce-4e49901dad1b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:10.273857 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631123cc-0887-4cd6-9580-d582727e3990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:10.274939 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631123cc-0887-4cd6-9580-d582727e3990 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:10.275372 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631123cc-0887-4cd6-9580-d582727e3990 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:10.275805 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631123cc-0887-4cd6-9580-d582727e3990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:10.301761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631123cc-0887-4cd6-9580-d582727e3990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:10.302204 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631123cc-0887-4cd6-9580-d582727e3990 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:10.318754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1407/2812] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:00:10.521098 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:10.864541 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae871d-c16d-4ae7-89e2-f8ff738c5b02 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:10.864823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae871d-c16d-4ae7-89e2-f8ff738c5b02 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:10.865100 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae871d-c16d-4ae7-89e2-f8ff738c5b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:10.885835 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-f5ae871d-c16d-4ae7-89e2-f8ff738c5b02 None None] MFA Rules not processed for user `bf8fecbb57394e2e8f5dabfd75ac6b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:10.887972 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f5ae871d-c16d-4ae7-89e2-f8ff738c5b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:10.946535 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1406/2813] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:11.615971 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631123cc-0887-4cd6-9580-d582727e3990 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:11.670532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:11.686050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631123cc-0887-4cd6-9580-d582727e3990 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:11.690915 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:11.691551 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:11.691837 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:11.692100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] PATH_INFO: `/v3/users/82bafb24a014425984618c745e7e0e90` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:11.693258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=82bafb24a014425984618c745e7e0e90)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:11.695275 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:12.038245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82bafb24a014425984618c745e7e0e90'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:12.052636 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82bafb24a014425984618c745e7e0e90'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:12.061428 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82bafb24a014425984618c745e7e0e90'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:12.133451 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82bafb24a014425984618c745e7e0e90'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:12.134219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82bafb24a014425984618c745e7e0e90'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:12.142323 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1408/2814] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:11 2022] DELETE /identity/v3/users/82bafb24a014425984618c745e7e0e90 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:12.147045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b8225-4647-4ec1-b900-4ef5dc8815d1 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:12.147604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b8225-4647-4ec1-b900-4ef5dc8815d1 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:12.147604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b8225-4647-4ec1-b900-4ef5dc8815d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:12.168779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-339b8225-4647-4ec1-b900-4ef5dc8815d1 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:12.170048 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-339b8225-4647-4ec1-b900-4ef5dc8815d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:12.240676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1407/2815] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:12.252235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:12.253738 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:12.320618 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842eb253-82ef-402d-a87b-99750aacb0ff admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:12.325938 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6794a8-6f08-4dc0-8320-c144c0f20adc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:12.327639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6794a8-6f08-4dc0-8320-c144c0f20adc service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:12.328100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6794a8-6f08-4dc0-8320-c144c0f20adc service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:12.328473 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6794a8-6f08-4dc0-8320-c144c0f20adc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:12.341631 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6794a8-6f08-4dc0-8320-c144c0f20adc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:12.343356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6794a8-6f08-4dc0-8320-c144c0f20adc service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:12.354716 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1409/2816] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:00:13.087068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b8225-4647-4ec1-b900-4ef5dc8815d1 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:13.087068 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-339b8225-4647-4ec1-b900-4ef5dc8815d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:13.099543 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b8225-4647-4ec1-b900-4ef5dc8815d1 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:13.108816 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:13.109975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:13.110336 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:13.110835 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] PATH_INFO: `/v3/projects/29197913955b4bdca450928038af981d` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:13.112951 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=29197913955b4bdca450928038af981d)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:13.116119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:13.328468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29197913955b4bdca450928038af981d', 'request_id': 'req-c9555623-6874-4e83-b8e7-8820dbbe1eba'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:13.477122 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29197913955b4bdca450928038af981d', 'request_id': 'req-c9555623-6874-4e83-b8e7-8820dbbe1eba'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:13.491836 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29197913955b4bdca450928038af981d', 'request_id': 'req-c9555623-6874-4e83-b8e7-8820dbbe1eba'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:13.518998 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] The token cache is being invalidate because project 29197913955b4bdca450928038af981d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:13.519424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:13.520507 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:13.523392 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1408/2817] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:13 2022] DELETE /identity/v3/projects/29197913955b4bdca450928038af981d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:13.531795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6794a8-6f08-4dc0-8320-c144c0f20adc service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:13.545819 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6794a8-6f08-4dc0-8320-c144c0f20adc service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:13.654097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7874a9-e090-4a46-b7e2-527b0739ca13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:13.654692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7874a9-e090-4a46-b7e2-527b0739ca13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:13.655018 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7874a9-e090-4a46-b7e2-527b0739ca13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:13.655310 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7874a9-e090-4a46-b7e2-527b0739ca13 admin admin] PATH_INFO: `/v3/users/8ac914bb88044a3a97e1b2d56fdd35d0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:13.656633 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7874a9-e090-4a46-b7e2-527b0739ca13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ac914bb88044a3a97e1b2d56fdd35d0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:13.659400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7874a9-e090-4a46-b7e2-527b0739ca13 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:13.705985 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:14.032848 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dc7874a9-e090-4a46-b7e2-527b0739ca13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ac914bb88044a3a97e1b2d56fdd35d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:14.047392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dc7874a9-e090-4a46-b7e2-527b0739ca13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ac914bb88044a3a97e1b2d56fdd35d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:14.055274 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dc7874a9-e090-4a46-b7e2-527b0739ca13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ac914bb88044a3a97e1b2d56fdd35d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:14.219358 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dc7874a9-e090-4a46-b7e2-527b0739ca13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ac914bb88044a3a97e1b2d56fdd35d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:14.220480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-dc7874a9-e090-4a46-b7e2-527b0739ca13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ac914bb88044a3a97e1b2d56fdd35d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:14.231729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1410/2818] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:13 2022] DELETE /identity/v3/users/8ac914bb88044a3a97e1b2d56fdd35d0 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:14.783755 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:14.786201 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:14.813458 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9555623-6874-4e83-b8e7-8820dbbe1eba admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:14.864251 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:14.912264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:14.913392 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:14.913671 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:14.913941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] PATH_INFO: `/v3/projects/dd70dd977b154b65976049bfb8ba24bf` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:14.915287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd70dd977b154b65976049bfb8ba24bf)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:14.917200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:15.059955 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd70dd977b154b65976049bfb8ba24bf', 'request_id': 'req-76e2d77c-7e52-46ab-884b-94f812169e99'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:15.183866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd70dd977b154b65976049bfb8ba24bf', 'request_id': 'req-76e2d77c-7e52-46ab-884b-94f812169e99'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:15.183866 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd70dd977b154b65976049bfb8ba24bf', 'request_id': 'req-76e2d77c-7e52-46ab-884b-94f812169e99'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:15.193878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] The token cache is being invalidate because project dd70dd977b154b65976049bfb8ba24bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:15.194175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:15.195155 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-76e2d77c-7e52-46ab-884b-94f812169e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:15.198159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1409/2819] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:14 2022] DELETE /identity/v3/projects/dd70dd977b154b65976049bfb8ba24bf => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:15.207148 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1a068-49c2-4ba2-800f-40eaf693509f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:15.213137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1a068-49c2-4ba2-800f-40eaf693509f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:15.215760 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1a068-49c2-4ba2-800f-40eaf693509f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:15.287990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7fd1a068-49c2-4ba2-800f-40eaf693509f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:15.290544 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7fd1a068-49c2-4ba2-800f-40eaf693509f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:15.434363 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1411/2820] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:15.442769 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512ab9e-e7fd-4e96-98f6-ec0b007e4579 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:15.443056 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512ab9e-e7fd-4e96-98f6-ec0b007e4579 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:15.443328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512ab9e-e7fd-4e96-98f6-ec0b007e4579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:15.465106 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-2512ab9e-e7fd-4e96-98f6-ec0b007e4579 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:15.466997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2512ab9e-e7fd-4e96-98f6-ec0b007e4579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:15.481351 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1410/2821] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:15.486314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1a068-49c2-4ba2-800f-40eaf693509f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:15.509905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1a068-49c2-4ba2-800f-40eaf693509f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:15.521568 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5ff66-ce48-4f2e-8aa3-09d7884a9d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:15.522872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5ff66-ce48-4f2e-8aa3-09d7884a9d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:15.523545 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5ff66-ce48-4f2e-8aa3-09d7884a9d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:15.524256 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5ff66-ce48-4f2e-8aa3-09d7884a9d1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:15.525733 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5ff66-ce48-4f2e-8aa3-09d7884a9d1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:00:15.526526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5ff66-ce48-4f2e-8aa3-09d7884a9d1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:15.531165 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5ff66-ce48-4f2e-8aa3-09d7884a9d1e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:15.563850 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1412/2822] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 30 00:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:00:15.564657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512ab9e-e7fd-4e96-98f6-ec0b007e4579 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:15.577946 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512ab9e-e7fd-4e96-98f6-ec0b007e4579 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:15.587061 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8763a3-ac76-47e8-88f4-6db12938c3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:15.588175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8763a3-ac76-47e8-88f4-6db12938c3df admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:15.596806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8763a3-ac76-47e8-88f4-6db12938c3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:15.596806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8763a3-ac76-47e8-88f4-6db12938c3df admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:15.596806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8763a3-ac76-47e8-88f4-6db12938c3df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:15.596806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8763a3-ac76-47e8-88f4-6db12938c3df admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:15.754377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1411/2823] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 30 00:00:15 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:00:15.759519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5ff66-ce48-4f2e-8aa3-09d7884a9d1e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:15.774755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5ff66-ce48-4f2e-8aa3-09d7884a9d1e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:15.843780 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0f508-873f-498d-8924-3c62d47e9719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:15.844683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0f508-873f-498d-8924-3c62d47e9719 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:15.845151 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0f508-873f-498d-8924-3c62d47e9719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:15.845747 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0f508-873f-498d-8924-3c62d47e9719 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:15.846595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0f508-873f-498d-8924-3c62d47e9719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:15.849050 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0f508-873f-498d-8924-3c62d47e9719 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:16.042601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1413/2824] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 30 00:00:15 2022] POST /identity/v3/users => generated 290 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:16.048545 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8763a3-ac76-47e8-88f4-6db12938c3df admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:16.060600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8763a3-ac76-47e8-88f4-6db12938c3df admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:16.065688 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110104-9f8a-40ca-bf4c-1d8faff76cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:16.066310 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110104-9f8a-40ca-bf4c-1d8faff76cfb admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:16.066621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110104-9f8a-40ca-bf4c-1d8faff76cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:16.066895 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110104-9f8a-40ca-bf4c-1d8faff76cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:16.067554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d110104-9f8a-40ca-bf4c-1d8faff76cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:16.069195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d110104-9f8a-40ca-bf4c-1d8faff76cfb admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:16.081120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1412/2825] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:00:16.085140 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0f508-873f-498d-8924-3c62d47e9719 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:16.098825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0f508-873f-498d-8924-3c62d47e9719 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:16.102826 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3f38c-32fb-47ed-8548-7e5bbb0e9955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:16.103502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3f38c-32fb-47ed-8548-7e5bbb0e9955 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:16.103788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3f38c-32fb-47ed-8548-7e5bbb0e9955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:16.104061 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3f38c-32fb-47ed-8548-7e5bbb0e9955 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:16.104723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3f38c-32fb-47ed-8548-7e5bbb0e9955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:16.107413 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3f38c-32fb-47ed-8548-7e5bbb0e9955 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:16.108919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:16.120303 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1414/2826] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:00:16.125277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110104-9f8a-40ca-bf4c-1d8faff76cfb admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:16.138639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110104-9f8a-40ca-bf4c-1d8faff76cfb admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:16.145989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccf6b7-0fd6-4d0a-80b8-fbca315af8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:16.147459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccf6b7-0fd6-4d0a-80b8-fbca315af8cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:16.147886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccf6b7-0fd6-4d0a-80b8-fbca315af8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:16.148485 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccf6b7-0fd6-4d0a-80b8-fbca315af8cb admin admin] PATH_INFO: `/v3/projects/95513811e48445f8be40b05dfda8543c/users/361508d150d443f9a6561d2ec545396b/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:16.176432 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccf6b7-0fd6-4d0a-80b8-fbca315af8cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=95513811e48445f8be40b05dfda8543c, user_id=361508d150d443f9a6561d2ec545396b, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:16.182250 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccf6b7-0fd6-4d0a-80b8-fbca315af8cb admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:16.295240 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1413/2827] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:16 2022] PUT /identity/v3/projects/95513811e48445f8be40b05dfda8543c/users/361508d150d443f9a6561d2ec545396b/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:16.307886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eb894-46f7-42a8-8055-967880ec4f73 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:16.308496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eb894-46f7-42a8-8055-967880ec4f73 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:16.308496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eb894-46f7-42a8-8055-967880ec4f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:16.340721 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-249eb894-46f7-42a8-8055-967880ec4f73 None None] MFA Rules not processed for user `361508d150d443f9a6561d2ec545396b`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:16.342135 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-249eb894-46f7-42a8-8055-967880ec4f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:16.424129 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1415/2828] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:16.434922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccf6b7-0fd6-4d0a-80b8-fbca315af8cb admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:16.560822 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:16.574273 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccf6b7-0fd6-4d0a-80b8-fbca315af8cb admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:16.580831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd373a24-c5d7-41ad-83a9-ad882b2c3429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:16.581636 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd373a24-c5d7-41ad-83a9-ad882b2c3429 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:16.581911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd373a24-c5d7-41ad-83a9-ad882b2c3429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:16.582178 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd373a24-c5d7-41ad-83a9-ad882b2c3429 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:16.583074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd373a24-c5d7-41ad-83a9-ad882b2c3429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:16.585040 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd373a24-c5d7-41ad-83a9-ad882b2c3429 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:16.731019 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1414/2829] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 30 00:00:16 2022] POST /identity/v3/projects => generated 381 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:16.735210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249eb894-46f7-42a8-8055-967880ec4f73 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:16.747074 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249eb894-46f7-42a8-8055-967880ec4f73 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:16.834304 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b5d21-43af-456a-968b-b3c6c40b6cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:16.835109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b5d21-43af-456a-968b-b3c6c40b6cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:16.835396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b5d21-43af-456a-968b-b3c6c40b6cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:16.836168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b5d21-43af-456a-968b-b3c6c40b6cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:16.836870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b5d21-43af-456a-968b-b3c6c40b6cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:16.839316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b5d21-43af-456a-968b-b3c6c40b6cc7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:16.963573 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1416/2830] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 30 00:00:16 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:00:16.970693 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd373a24-c5d7-41ad-83a9-ad882b2c3429 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:16.981792 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd373a24-c5d7-41ad-83a9-ad882b2c3429 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:16.987169 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693f38-d520-4438-8171-d862acb524b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:16.987729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693f38-d520-4438-8171-d862acb524b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:16.987943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693f38-d520-4438-8171-d862acb524b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:16.988315 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693f38-d520-4438-8171-d862acb524b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:16.988975 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f693f38-d520-4438-8171-d862acb524b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:16.990717 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f693f38-d520-4438-8171-d862acb524b6 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:17.002403 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1415/2831] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:00:17.006898 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b5d21-43af-456a-968b-b3c6c40b6cc7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:17.018450 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b5d21-43af-456a-968b-b3c6c40b6cc7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:17.023919 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed8c1b-8c51-4f46-9655-e1b5507bf741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:17.024754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed8c1b-8c51-4f46-9655-e1b5507bf741 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:17.025038 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed8c1b-8c51-4f46-9655-e1b5507bf741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:17.025314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed8c1b-8c51-4f46-9655-e1b5507bf741 admin admin] PATH_INFO: `/v3/projects/1ad9387176994b4484832ae050dc83d4/users/b2cb2cc26ad049898192a51510a1766e/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:17.046994 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed8c1b-8c51-4f46-9655-e1b5507bf741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad9387176994b4484832ae050dc83d4, user_id=b2cb2cc26ad049898192a51510a1766e, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:17.050143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed8c1b-8c51-4f46-9655-e1b5507bf741 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:17.172807 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1417/2832] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:17 2022] PUT /identity/v3/projects/1ad9387176994b4484832ae050dc83d4/users/b2cb2cc26ad049898192a51510a1766e/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:17.176696 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693f38-d520-4438-8171-d862acb524b6 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:17.188199 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693f38-d520-4438-8171-d862acb524b6 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:17.252926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b82af2-3860-48f6-9d73-9e2b2710337c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:17.253709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b82af2-3860-48f6-9d73-9e2b2710337c admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:17.254013 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b82af2-3860-48f6-9d73-9e2b2710337c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:17.254365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b82af2-3860-48f6-9d73-9e2b2710337c admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:17.255102 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b82af2-3860-48f6-9d73-9e2b2710337c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:17.256771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b82af2-3860-48f6-9d73-9e2b2710337c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:17.267540 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1416/2833] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:00:17.271634 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed8c1b-8c51-4f46-9655-e1b5507bf741 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:17.283088 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed8c1b-8c51-4f46-9655-e1b5507bf741 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:17.288095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac1a95-577a-49bc-b9e2-bf36c3aaa05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:17.288988 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac1a95-577a-49bc-b9e2-bf36c3aaa05b admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:17.289261 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac1a95-577a-49bc-b9e2-bf36c3aaa05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:17.289532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac1a95-577a-49bc-b9e2-bf36c3aaa05b admin admin] PATH_INFO: `/v3/projects/1ad9387176994b4484832ae050dc83d4/users/b2cb2cc26ad049898192a51510a1766e/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:17.291535 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac1a95-577a-49bc-b9e2-bf36c3aaa05b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad9387176994b4484832ae050dc83d4, user_id=b2cb2cc26ad049898192a51510a1766e, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:17.294833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac1a95-577a-49bc-b9e2-bf36c3aaa05b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:17.309080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1418/2834] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:17 2022] PUT /identity/v3/projects/1ad9387176994b4484832ae050dc83d4/users/b2cb2cc26ad049898192a51510a1766e/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:00:17.319289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a54859-1338-4dcf-bf9c-555972b0b58f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:17.319857 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a54859-1338-4dcf-bf9c-555972b0b58f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:17.321319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a54859-1338-4dcf-bf9c-555972b0b58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:17.392042 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-67a54859-1338-4dcf-bf9c-555972b0b58f None None] MFA Rules not processed for user `b2cb2cc26ad049898192a51510a1766e`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:17.394182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-67a54859-1338-4dcf-bf9c-555972b0b58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:17.493869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1417/2835] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:17.505884 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac1a95-577a-49bc-b9e2-bf36c3aaa05b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:17.507832 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-eaac1a95-577a-49bc-b9e2-bf36c3aaa05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:17.578909 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:17.597719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac1a95-577a-49bc-b9e2-bf36c3aaa05b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:17.602292 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b0cf2-dee8-4898-a9fe-caed5b1fcb2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:17.602952 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b0cf2-dee8-4898-a9fe-caed5b1fcb2b service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:17.603234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b0cf2-dee8-4898-a9fe-caed5b1fcb2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:17.604522 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b0cf2-dee8-4898-a9fe-caed5b1fcb2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:17.615744 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b0cf2-dee8-4898-a9fe-caed5b1fcb2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:17.617607 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b0cf2-dee8-4898-a9fe-caed5b1fcb2b service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:17.632808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1419/2836] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:00:18.344777 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a54859-1338-4dcf-bf9c-555972b0b58f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:18.407504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:18.422360 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a54859-1338-4dcf-bf9c-555972b0b58f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:18.426984 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:18.427621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:18.427893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:18.428166 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] PATH_INFO: `/v3/users/361508d150d443f9a6561d2ec545396b` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:18.429432 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=361508d150d443f9a6561d2ec545396b)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:18.431410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:18.707667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361508d150d443f9a6561d2ec545396b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:18.734779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361508d150d443f9a6561d2ec545396b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:18.751473 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361508d150d443f9a6561d2ec545396b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:18.873414 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361508d150d443f9a6561d2ec545396b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:18.874153 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361508d150d443f9a6561d2ec545396b'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:18.881823 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1418/2837] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:18 2022] DELETE /identity/v3/users/361508d150d443f9a6561d2ec545396b => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:18.886613 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2391cc9-e4f1-4d01-b1a0-a020737989f8 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:18.887453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2391cc9-e4f1-4d01-b1a0-a020737989f8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:18.887453 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2391cc9-e4f1-4d01-b1a0-a020737989f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:18.909966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c2391cc9-e4f1-4d01-b1a0-a020737989f8 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:18.911355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c2391cc9-e4f1-4d01-b1a0-a020737989f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:18.986600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1420/2838] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:18.995935 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:18.997516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:19.087359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e202df0-bac1-47a5-a140-e72de3476c59 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:19.089468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e89224-fe56-4e72-88ab-09c9e42a7a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:19.090865 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e89224-fe56-4e72-88ab-09c9e42a7a99 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:19.091180 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e89224-fe56-4e72-88ab-09c9e42a7a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:19.091445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e89224-fe56-4e72-88ab-09c9e42a7a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:19.101737 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e89224-fe56-4e72-88ab-09c9e42a7a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:19.103319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e89224-fe56-4e72-88ab-09c9e42a7a99 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:19.114509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1419/2839] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:00:19.841106 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2391cc9-e4f1-4d01-b1a0-a020737989f8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:19.842695 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c2391cc9-e4f1-4d01-b1a0-a020737989f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:19.856463 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2391cc9-e4f1-4d01-b1a0-a020737989f8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:19.862045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:19.862625 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:19.862926 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:19.863188 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] PATH_INFO: `/v3/projects/95513811e48445f8be40b05dfda8543c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:19.864639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95513811e48445f8be40b05dfda8543c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:19.866600 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:20.043960 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95513811e48445f8be40b05dfda8543c', 'request_id': 'req-38443536-e8bc-4c1c-aed7-6406a4d2b417'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:20.049683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95513811e48445f8be40b05dfda8543c', 'request_id': 'req-38443536-e8bc-4c1c-aed7-6406a4d2b417'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:20.174122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95513811e48445f8be40b05dfda8543c', 'request_id': 'req-38443536-e8bc-4c1c-aed7-6406a4d2b417'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:20.188086 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] The token cache is being invalidate because project 95513811e48445f8be40b05dfda8543c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:20.188376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:20.189075 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:20.190574 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1421/2840] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:19 2022] DELETE /identity/v3/projects/95513811e48445f8be40b05dfda8543c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:20.195264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e89224-fe56-4e72-88ab-09c9e42a7a99 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:20.205897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e89224-fe56-4e72-88ab-09c9e42a7a99 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:20.275179 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe876b1a-8158-4053-8665-e4a01e1b9269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:20.275995 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe876b1a-8158-4053-8665-e4a01e1b9269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:20.276316 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe876b1a-8158-4053-8665-e4a01e1b9269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:20.276673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe876b1a-8158-4053-8665-e4a01e1b9269 admin admin] PATH_INFO: `/v3/users/b2cb2cc26ad049898192a51510a1766e` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:20.278627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe876b1a-8158-4053-8665-e4a01e1b9269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2cb2cc26ad049898192a51510a1766e)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:20.281147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe876b1a-8158-4053-8665-e4a01e1b9269 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:20.297767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:20.490879 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fe876b1a-8158-4053-8665-e4a01e1b9269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2cb2cc26ad049898192a51510a1766e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:20.504957 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fe876b1a-8158-4053-8665-e4a01e1b9269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2cb2cc26ad049898192a51510a1766e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:20.513775 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fe876b1a-8158-4053-8665-e4a01e1b9269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2cb2cc26ad049898192a51510a1766e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:20.591452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fe876b1a-8158-4053-8665-e4a01e1b9269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2cb2cc26ad049898192a51510a1766e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:20.592542 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-fe876b1a-8158-4053-8665-e4a01e1b9269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2cb2cc26ad049898192a51510a1766e'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:20.600451 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1420/2841] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:20 2022] DELETE /identity/v3/users/b2cb2cc26ad049898192a51510a1766e => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:21.284371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:21.287271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:21.323968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443536-e8bc-4c1c-aed7-6406a4d2b417 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:21.421644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:21.422771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:21.423258 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:21.423755 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] PATH_INFO: `/v3/projects/1ad9387176994b4484832ae050dc83d4` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:21.425794 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ad9387176994b4484832ae050dc83d4)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:21.429614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:21.439359 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:21.567340 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ad9387176994b4484832ae050dc83d4', 'request_id': 'req-81eedef3-2179-42c3-b092-71bc904bc4f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:21.573847 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ad9387176994b4484832ae050dc83d4', 'request_id': 'req-81eedef3-2179-42c3-b092-71bc904bc4f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:21.676479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ad9387176994b4484832ae050dc83d4', 'request_id': 'req-81eedef3-2179-42c3-b092-71bc904bc4f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:21.689833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] The token cache is being invalidate because project 1ad9387176994b4484832ae050dc83d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:21.690288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:21.691049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-81eedef3-2179-42c3-b092-71bc904bc4f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:21.692712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1422/2842] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:21 2022] DELETE /identity/v3/projects/1ad9387176994b4484832ae050dc83d4 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:21.698044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d99bd-f3b6-4eb5-bd5f-8578b246f225 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:21.698579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d99bd-f3b6-4eb5-bd5f-8578b246f225 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:21.698892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d99bd-f3b6-4eb5-bd5f-8578b246f225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:21.724520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-b22d99bd-f3b6-4eb5-bd5f-8578b246f225 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:21.725893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b22d99bd-f3b6-4eb5-bd5f-8578b246f225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:21.795231 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1421/2843] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:00:21.803384 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826320d-0efb-427e-8c38-fe389e02a7b9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:21.803669 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826320d-0efb-427e-8c38-fe389e02a7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:21.803968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826320d-0efb-427e-8c38-fe389e02a7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:21.825420 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1826320d-0efb-427e-8c38-fe389e02a7b9 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:21.826761 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1826320d-0efb-427e-8c38-fe389e02a7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:21.835724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1423/2844] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:21.840922 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d99bd-f3b6-4eb5-bd5f-8578b246f225 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:21.850971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d99bd-f3b6-4eb5-bd5f-8578b246f225 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:21.856346 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a95fcd-d392-402e-88c8-d3835a8ea78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:21.856961 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a95fcd-d392-402e-88c8-d3835a8ea78d admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:21.857239 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a95fcd-d392-402e-88c8-d3835a8ea78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:21.857526 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a95fcd-d392-402e-88c8-d3835a8ea78d admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:21.858193 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a95fcd-d392-402e-88c8-d3835a8ea78d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:00:21.858524 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a95fcd-d392-402e-88c8-d3835a8ea78d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:21.860114 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a95fcd-d392-402e-88c8-d3835a8ea78d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:21.875480 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1422/2845] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 30 00:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:00:21.881277 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826320d-0efb-427e-8c38-fe389e02a7b9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:21.896216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826320d-0efb-427e-8c38-fe389e02a7b9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:21.904410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb139f-fe51-4f84-b538-4bf17b1cd1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:21.905022 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb139f-fe51-4f84-b538-4bf17b1cd1e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:21.905301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb139f-fe51-4f84-b538-4bf17b1cd1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:21.907179 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb139f-fe51-4f84-b538-4bf17b1cd1e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:21.907179 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb139f-fe51-4f84-b538-4bf17b1cd1e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:21.908243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb139f-fe51-4f84-b538-4bf17b1cd1e2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:22.018399 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1424/2846] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 30 00:00:21 2022] POST /identity/v3/projects => generated 361 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:22.023046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a95fcd-d392-402e-88c8-d3835a8ea78d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:22.034343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a95fcd-d392-402e-88c8-d3835a8ea78d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:22.110009 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ab11e-3d58-4d3e-a107-8cd6b2a9819d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:22.110635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ab11e-3d58-4d3e-a107-8cd6b2a9819d admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:22.110913 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ab11e-3d58-4d3e-a107-8cd6b2a9819d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:22.111184 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ab11e-3d58-4d3e-a107-8cd6b2a9819d admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:22.111831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ab11e-3d58-4d3e-a107-8cd6b2a9819d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:22.113774 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ab11e-3d58-4d3e-a107-8cd6b2a9819d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:22.237754 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1423/2847] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 30 00:00:22 2022] POST /identity/v3/users => generated 281 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:22.243042 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb139f-fe51-4f84-b538-4bf17b1cd1e2 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:22.254198 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb139f-fe51-4f84-b538-4bf17b1cd1e2 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:22.263841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d755969-9efa-42ed-908b-19a9aa657c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:22.264624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d755969-9efa-42ed-908b-19a9aa657c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:22.264907 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d755969-9efa-42ed-908b-19a9aa657c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:22.265172 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d755969-9efa-42ed-908b-19a9aa657c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:22.265817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d755969-9efa-42ed-908b-19a9aa657c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:22.267467 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d755969-9efa-42ed-908b-19a9aa657c63 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:22.281770 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1425/2848] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:00:22.286277 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ab11e-3d58-4d3e-a107-8cd6b2a9819d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:22.297424 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ab11e-3d58-4d3e-a107-8cd6b2a9819d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:22.303893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd6762-66ee-4f18-91ad-5f0aa6a63733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:22.309939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd6762-66ee-4f18-91ad-5f0aa6a63733 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:22.309939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd6762-66ee-4f18-91ad-5f0aa6a63733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:22.309939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd6762-66ee-4f18-91ad-5f0aa6a63733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:22.309939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd6762-66ee-4f18-91ad-5f0aa6a63733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:22.309939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd6762-66ee-4f18-91ad-5f0aa6a63733 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:22.309939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:22.322158 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1424/2849] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:00:22.327693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d755969-9efa-42ed-908b-19a9aa657c63 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:22.338590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d755969-9efa-42ed-908b-19a9aa657c63 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:22.343305 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64e40e-5f66-4260-86ce-65e3710a5ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:22.343980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64e40e-5f66-4260-86ce-65e3710a5ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:22.344314 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64e40e-5f66-4260-86ce-65e3710a5ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:22.344597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64e40e-5f66-4260-86ce-65e3710a5ea8 admin admin] PATH_INFO: `/v3/projects/3e4ed96d6c7843a794b3afe87df134d6/users/6d3f80440c934e369fdc5d473897f38a/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:22.361040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64e40e-5f66-4260-86ce-65e3710a5ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4ed96d6c7843a794b3afe87df134d6, user_id=6d3f80440c934e369fdc5d473897f38a, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:22.363771 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64e40e-5f66-4260-86ce-65e3710a5ea8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:22.482745 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1426/2850] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:22 2022] PUT /identity/v3/projects/3e4ed96d6c7843a794b3afe87df134d6/users/6d3f80440c934e369fdc5d473897f38a/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:00:22.494854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4f3d0-f6de-4f13-b727-8434a0e3947b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:22.495141 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4f3d0-f6de-4f13-b727-8434a0e3947b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:22.495423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4f3d0-f6de-4f13-b727-8434a0e3947b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:22.526841 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-02a4f3d0-f6de-4f13-b727-8434a0e3947b None None] MFA Rules not processed for user `6d3f80440c934e369fdc5d473897f38a`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:22.528112 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-02a4f3d0-f6de-4f13-b727-8434a0e3947b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:22.601041 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1425/2851] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:22.645288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64e40e-5f66-4260-86ce-65e3710a5ea8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:22.647045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2c64e40e-5f66-4260-86ce-65e3710a5ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:22.711174 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64e40e-5f66-4260-86ce-65e3710a5ea8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:22.714836 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca19772-1a1c-4a6b-8cea-e1a1564b0534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:22.716788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca19772-1a1c-4a6b-8cea-e1a1564b0534 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:22.716788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca19772-1a1c-4a6b-8cea-e1a1564b0534 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:22.716788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca19772-1a1c-4a6b-8cea-e1a1564b0534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:22.725729 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca19772-1a1c-4a6b-8cea-e1a1564b0534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:22.727242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca19772-1a1c-4a6b-8cea-e1a1564b0534 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:22.736622 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1427/2852] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:00:22.751709 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4f3d0-f6de-4f13-b727-8434a0e3947b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:22.837707 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4f3d0-f6de-4f13-b727-8434a0e3947b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:22.843411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:22.844047 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:22.844321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:22.844588 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] PATH_INFO: `/v3/users/6d3f80440c934e369fdc5d473897f38a` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:22.845766 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d3f80440c934e369fdc5d473897f38a)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:22.847756 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:23.151604 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d3f80440c934e369fdc5d473897f38a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:23.171785 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d3f80440c934e369fdc5d473897f38a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:23.179708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d3f80440c934e369fdc5d473897f38a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:23.295433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d3f80440c934e369fdc5d473897f38a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:23.297055 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d3f80440c934e369fdc5d473897f38a'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:23.318119 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1426/2853] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:22 2022] DELETE /identity/v3/users/6d3f80440c934e369fdc5d473897f38a => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:23.325115 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b41531-81ce-44fd-9c66-4fecd92c8bd3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:23.326149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b41531-81ce-44fd-9c66-4fecd92c8bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:23.326149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b41531-81ce-44fd-9c66-4fecd92c8bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:23.373212 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e0b41531-81ce-44fd-9c66-4fecd92c8bd3 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:23.374517 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e0b41531-81ce-44fd-9c66-4fecd92c8bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:23.424449 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:23.477768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1428/2854] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:23.488599 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:23.490049 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:23.553128 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c3f18-1162-4d98-a874-b5149538864b admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:23.558185 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741d155-b96b-4038-8f36-25e8400535d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:23.558931 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741d155-b96b-4038-8f36-25e8400535d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:23.559328 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741d155-b96b-4038-8f36-25e8400535d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:23.559601 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741d155-b96b-4038-8f36-25e8400535d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:23.570730 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741d155-b96b-4038-8f36-25e8400535d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:23.572237 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741d155-b96b-4038-8f36-25e8400535d6 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:23.583001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1427/2855] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:00:23.711474 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:24.171506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b41531-81ce-44fd-9c66-4fecd92c8bd3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:24.173400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e0b41531-81ce-44fd-9c66-4fecd92c8bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:24.189578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b41531-81ce-44fd-9c66-4fecd92c8bd3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:24.201342 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:24.202000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:24.202279 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:24.202589 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] PATH_INFO: `/v3/projects/3e4ed96d6c7843a794b3afe87df134d6` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:24.203856 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e4ed96d6c7843a794b3afe87df134d6)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:24.205833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:24.350345 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e4ed96d6c7843a794b3afe87df134d6', 'request_id': 'req-d3686e66-c55c-4a12-ada7-b155d08d0b10'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:24.358991 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e4ed96d6c7843a794b3afe87df134d6', 'request_id': 'req-d3686e66-c55c-4a12-ada7-b155d08d0b10'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:24.446723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e4ed96d6c7843a794b3afe87df134d6', 'request_id': 'req-d3686e66-c55c-4a12-ada7-b155d08d0b10'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:24.466016 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] The token cache is being invalidate because project 3e4ed96d6c7843a794b3afe87df134d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:24.467173 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:24.468489 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d3686e66-c55c-4a12-ada7-b155d08d0b10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:24.471608 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1429/2856] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:24 2022] DELETE /identity/v3/projects/3e4ed96d6c7843a794b3afe87df134d6 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:24.478763 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c4c4-d681-452b-b25f-682f08b826b0 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:24.479313 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c4c4-d681-452b-b25f-682f08b826b0 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:24.479793 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c4c4-d681-452b-b25f-682f08b826b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:24.501466 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-5440c4c4-d681-452b-b25f-682f08b826b0 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:24.502837 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-5440c4c4-d681-452b-b25f-682f08b826b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:24.604897 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1428/2857] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:24.614990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e48f84-2178-44ae-a6b8-cfa7ccf05688 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:24.616558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e48f84-2178-44ae-a6b8-cfa7ccf05688 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:24.617080 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e48f84-2178-44ae-a6b8-cfa7ccf05688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:24.653087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c3e48f84-2178-44ae-a6b8-cfa7ccf05688 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:24.654597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c3e48f84-2178-44ae-a6b8-cfa7ccf05688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:24.665073 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1430/2858] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:24.670001 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440c4c4-d681-452b-b25f-682f08b826b0 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:24.680612 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440c4c4-d681-452b-b25f-682f08b826b0 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:24.686657 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02faa10-6be1-4767-be72-db197afc4064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:24.687290 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02faa10-6be1-4767-be72-db197afc4064 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:24.687565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02faa10-6be1-4767-be72-db197afc4064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:24.688459 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02faa10-6be1-4767-be72-db197afc4064 admin admin] PATH_INFO: `/v3/domains` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:24.688662 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02faa10-6be1-4767-be72-db197afc4064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81853) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:00:24.688826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02faa10-6be1-4767-be72-db197afc4064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:24.691415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02faa10-6be1-4767-be72-db197afc4064 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:24.708990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1429/2859] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 30 00:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:00:24.713980 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e48f84-2178-44ae-a6b8-cfa7ccf05688 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:24.725011 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e48f84-2178-44ae-a6b8-cfa7ccf05688 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:24.729947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434382c-eb9e-4ba0-a002-449047e9672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:24.730632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434382c-eb9e-4ba0-a002-449047e9672b admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:24.730913 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434382c-eb9e-4ba0-a002-449047e9672b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:24.731230 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434382c-eb9e-4ba0-a002-449047e9672b admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:24.731810 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434382c-eb9e-4ba0-a002-449047e9672b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:24.733727 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434382c-eb9e-4ba0-a002-449047e9672b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:24.862353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1431/2860] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 30 00:00:24 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:24.866714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02faa10-6be1-4767-be72-db197afc4064 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:24.877953 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02faa10-6be1-4767-be72-db197afc4064 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:24.943387 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaa165-52de-4a02-bcf3-1de5fd40f98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:24.944012 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaa165-52de-4a02-bcf3-1de5fd40f98f admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:24.944420 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaa165-52de-4a02-bcf3-1de5fd40f98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:24.944652 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaa165-52de-4a02-bcf3-1de5fd40f98f admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:24.945419 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbaa165-52de-4a02-bcf3-1de5fd40f98f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:24.947565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbaa165-52de-4a02-bcf3-1de5fd40f98f admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:25.141329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1430/2861] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 30 00:00:24 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:25.146220 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434382c-eb9e-4ba0-a002-449047e9672b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:25.162210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434382c-eb9e-4ba0-a002-449047e9672b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:25.168944 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20093f-269e-454f-a0f6-9c01acaee6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:25.169795 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20093f-269e-454f-a0f6-9c01acaee6ba admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:25.170717 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20093f-269e-454f-a0f6-9c01acaee6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:25.171085 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20093f-269e-454f-a0f6-9c01acaee6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:25.172229 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20093f-269e-454f-a0f6-9c01acaee6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:25.174091 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20093f-269e-454f-a0f6-9c01acaee6ba admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:25.189004 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1432/2862] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:00:25.195834 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaa165-52de-4a02-bcf3-1de5fd40f98f admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:25.213283 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaa165-52de-4a02-bcf3-1de5fd40f98f admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:25.220828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007a9ff-1898-4dbd-8af5-cddc872a58ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:25.221494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007a9ff-1898-4dbd-8af5-cddc872a58ee admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:25.221771 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007a9ff-1898-4dbd-8af5-cddc872a58ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:25.222081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007a9ff-1898-4dbd-8af5-cddc872a58ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:25.223058 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007a9ff-1898-4dbd-8af5-cddc872a58ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:25.225827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007a9ff-1898-4dbd-8af5-cddc872a58ee admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:25.244767 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1431/2863] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:00:25.258405 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20093f-269e-454f-a0f6-9c01acaee6ba admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:25.269418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20093f-269e-454f-a0f6-9c01acaee6ba admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:25.275068 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ba471-8cd2-4b9b-89fd-0b4f07463504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:25.275759 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ba471-8cd2-4b9b-89fd-0b4f07463504 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:25.276136 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ba471-8cd2-4b9b-89fd-0b4f07463504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:25.276400 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ba471-8cd2-4b9b-89fd-0b4f07463504 admin admin] PATH_INFO: `/v3/projects/c7965c208d7f4436a1f2ff0e6acc5414/users/c4b5f5170183470097a04a2fc2ccc8c1/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:25.295604 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ba471-8cd2-4b9b-89fd-0b4f07463504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7965c208d7f4436a1f2ff0e6acc5414, user_id=c4b5f5170183470097a04a2fc2ccc8c1, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:25.301817 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ba471-8cd2-4b9b-89fd-0b4f07463504 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:25.460902 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1433/2864] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:25 2022] PUT /identity/v3/projects/c7965c208d7f4436a1f2ff0e6acc5414/users/c4b5f5170183470097a04a2fc2ccc8c1/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:00:25.471546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44487-f043-459f-a71a-abbf33eb3c28 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:25.471852 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44487-f043-459f-a71a-abbf33eb3c28 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:25.472083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44487-f043-459f-a71a-abbf33eb3c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:25.504659 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:25.508654 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-76e44487-f043-459f-a71a-abbf33eb3c28 None None] MFA Rules not processed for user `c4b5f5170183470097a04a2fc2ccc8c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:25.510025 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-76e44487-f043-459f-a71a-abbf33eb3c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:25.589388 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1432/2865] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:25.594387 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ba471-8cd2-4b9b-89fd-0b4f07463504 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:25.651245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:25.662168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ba471-8cd2-4b9b-89fd-0b4f07463504 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:25.666968 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa65888-3d8a-498e-b201-512feb466654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:25.667580 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa65888-3d8a-498e-b201-512feb466654 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:25.667859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa65888-3d8a-498e-b201-512feb466654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:25.668661 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa65888-3d8a-498e-b201-512feb466654 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:25.669385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa65888-3d8a-498e-b201-512feb466654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:25.671396 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa65888-3d8a-498e-b201-512feb466654 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:25.831092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1434/2866] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 30 00:00:25 2022] POST /identity/v3/projects => generated 373 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:25.832740 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e44487-f043-459f-a71a-abbf33eb3c28 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:25.845464 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e44487-f043-459f-a71a-abbf33eb3c28 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:25.910030 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4d6f3-9b21-4ed5-9c61-f1d8dda93910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:25.910581 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4d6f3-9b21-4ed5-9c61-f1d8dda93910 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:25.911095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4d6f3-9b21-4ed5-9c61-f1d8dda93910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:25.911359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4d6f3-9b21-4ed5-9c61-f1d8dda93910 admin admin] PATH_INFO: `/v3/users` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:25.912037 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4d6f3-9b21-4ed5-9c61-f1d8dda93910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:25.913958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4d6f3-9b21-4ed5-9c61-f1d8dda93910 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:26.024986 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1433/2867] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 30 00:00:25 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:00:26.030914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa65888-3d8a-498e-b201-512feb466654 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:26.041835 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa65888-3d8a-498e-b201-512feb466654 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:26.047329 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3247cb0-7976-471e-82a7-adb5ca74ec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:26.048040 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3247cb0-7976-471e-82a7-adb5ca74ec0e admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:26.048341 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3247cb0-7976-471e-82a7-adb5ca74ec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:26.048578 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3247cb0-7976-471e-82a7-adb5ca74ec0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:26.049210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3247cb0-7976-471e-82a7-adb5ca74ec0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:26.050883 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3247cb0-7976-471e-82a7-adb5ca74ec0e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:26.063334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1435/2868] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:00:26.070095 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4d6f3-9b21-4ed5-9c61-f1d8dda93910 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:26.088675 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4d6f3-9b21-4ed5-9c61-f1d8dda93910 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:26.095210 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbfd22-2ff0-4032-b596-9694ba5135c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:26.095892 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbfd22-2ff0-4032-b596-9694ba5135c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:26.096329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbfd22-2ff0-4032-b596-9694ba5135c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:26.096704 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbfd22-2ff0-4032-b596-9694ba5135c4 admin admin] PATH_INFO: `/v3/projects/47078bf69d814e569337b0300a7001aa/users/9dd434c375324d3a828fa5265c5bc2c8/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:26.115067 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dbfd22-2ff0-4032-b596-9694ba5135c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47078bf69d814e569337b0300a7001aa, user_id=9dd434c375324d3a828fa5265c5bc2c8, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:26.117992 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dbfd22-2ff0-4032-b596-9694ba5135c4 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:26.226982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1434/2869] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:26 2022] PUT /identity/v3/projects/47078bf69d814e569337b0300a7001aa/users/9dd434c375324d3a828fa5265c5bc2c8/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:26.231243 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3247cb0-7976-471e-82a7-adb5ca74ec0e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:26.242069 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3247cb0-7976-471e-82a7-adb5ca74ec0e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:26.312193 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fdbb8-eee0-40fe-bf58-f993fee952f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:26.312803 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fdbb8-eee0-40fe-bf58-f993fee952f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:26.313076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fdbb8-eee0-40fe-bf58-f993fee952f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:26.313351 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fdbb8-eee0-40fe-bf58-f993fee952f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:26.314000 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fdbb8-eee0-40fe-bf58-f993fee952f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:26.315649 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fdbb8-eee0-40fe-bf58-f993fee952f7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:26.326905 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1436/2870] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:00:26.334494 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbfd22-2ff0-4032-b596-9694ba5135c4 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:26.346610 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbfd22-2ff0-4032-b596-9694ba5135c4 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:26.353916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac43ab-6dcf-43c9-9c47-0cd8b6d3ce9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:26.354607 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac43ab-6dcf-43c9-9c47-0cd8b6d3ce9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:26.354885 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac43ab-6dcf-43c9-9c47-0cd8b6d3ce9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:26.355202 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac43ab-6dcf-43c9-9c47-0cd8b6d3ce9e admin admin] PATH_INFO: `/v3/projects/47078bf69d814e569337b0300a7001aa/users/9dd434c375324d3a828fa5265c5bc2c8/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:26.357716 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac43ab-6dcf-43c9-9c47-0cd8b6d3ce9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47078bf69d814e569337b0300a7001aa, user_id=9dd434c375324d3a828fa5265c5bc2c8, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:26.360433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac43ab-6dcf-43c9-9c47-0cd8b6d3ce9e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:26.371391 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1435/2871] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:00:26 2022] PUT /identity/v3/projects/47078bf69d814e569337b0300a7001aa/users/9dd434c375324d3a828fa5265c5bc2c8/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:26.441356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30dd4d-44cc-4912-afb5-a6683c3adee7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:26.441724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30dd4d-44cc-4912-afb5-a6683c3adee7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:26.442008 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30dd4d-44cc-4912-afb5-a6683c3adee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:26.476928 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ce30dd4d-44cc-4912-afb5-a6683c3adee7 None None] MFA Rules not processed for user `9dd434c375324d3a828fa5265c5bc2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:26.478276 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ce30dd4d-44cc-4912-afb5-a6683c3adee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:26.571247 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1437/2872] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:26.585590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac43ab-6dcf-43c9-9c47-0cd8b6d3ce9e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:26.588381 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-2dac43ab-6dcf-43c9-9c47-0cd8b6d3ce9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:26.692435 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac43ab-6dcf-43c9-9c47-0cd8b6d3ce9e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:26.696032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63c68b-38cc-4ab6-8efd-e16fb8962bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:26.696630 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c68b-38cc-4ab6-8efd-e16fb8962bdd service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:26.696909 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c68b-38cc-4ab6-8efd-e16fb8962bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:26.697227 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c68b-38cc-4ab6-8efd-e16fb8962bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:26.706484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:26.760108 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63c68b-38cc-4ab6-8efd-e16fb8962bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:26.761558 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63c68b-38cc-4ab6-8efd-e16fb8962bdd service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:26.771829 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1436/2873] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:00:30.344289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30dd4d-44cc-4912-afb5-a6683c3adee7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:30.414005 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:30.442268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30dd4d-44cc-4912-afb5-a6683c3adee7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:30.450122 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee29db-93f5-4843-9443-aef024592c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:30.451285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee29db-93f5-4843-9443-aef024592c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:30.451768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee29db-93f5-4843-9443-aef024592c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:30.452103 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee29db-93f5-4843-9443-aef024592c26 admin admin] PATH_INFO: `/v3/users/c4b5f5170183470097a04a2fc2ccc8c1` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:30.454221 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee29db-93f5-4843-9443-aef024592c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4b5f5170183470097a04a2fc2ccc8c1)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:30.457895 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee29db-93f5-4843-9443-aef024592c26 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:30.524423 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca7899-7400-48c8-a48c-4cf998529a40 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:30.525343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca7899-7400-48c8-a48c-4cf998529a40 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:30.525343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca7899-7400-48c8-a48c-4cf998529a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:30.561908 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-4cca7899-7400-48c8-a48c-4cf998529a40 None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:30.563183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4cca7899-7400-48c8-a48c-4cf998529a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:30.639440 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1437/2874] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 30 00:00:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:00:30.683355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-41ee29db-93f5-4843-9443-aef024592c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4b5f5170183470097a04a2fc2ccc8c1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:30.698034 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-41ee29db-93f5-4843-9443-aef024592c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4b5f5170183470097a04a2fc2ccc8c1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:30.705398 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-41ee29db-93f5-4843-9443-aef024592c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4b5f5170183470097a04a2fc2ccc8c1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:30.817851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca7899-7400-48c8-a48c-4cf998529a40 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:30.819392 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4cca7899-7400-48c8-a48c-4cf998529a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:30.825647 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-41ee29db-93f5-4843-9443-aef024592c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4b5f5170183470097a04a2fc2ccc8c1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:30.826312 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-41ee29db-93f5-4843-9443-aef024592c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4b5f5170183470097a04a2fc2ccc8c1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:30.838318 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1438/2875] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:30 2022] DELETE /identity/v3/users/c4b5f5170183470097a04a2fc2ccc8c1 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:30.843442 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da1649-3192-4cbd-9ef3-36ac9628f4ba None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:30.846271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da1649-3192-4cbd-9ef3-36ac9628f4ba None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:30.846271 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da1649-3192-4cbd-9ef3-36ac9628f4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:30.871783 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-96da1649-3192-4cbd-9ef3-36ac9628f4ba None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:30.873089 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-96da1649-3192-4cbd-9ef3-36ac9628f4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:30.931625 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca7899-7400-48c8-a48c-4cf998529a40 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:30.947611 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1439/2876] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:00:30.959369 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da1649-3192-4cbd-9ef3-36ac9628f4ba None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:30.960820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-96da1649-3192-4cbd-9ef3-36ac9628f4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:30.993017 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:31.022827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c8cd5-bd9c-4fcb-a26a-eb0d244710de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:31.023849 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c8cd5-bd9c-4fcb-a26a-eb0d244710de service swift] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:31.024350 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c8cd5-bd9c-4fcb-a26a-eb0d244710de service swift] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:31.024854 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c8cd5-bd9c-4fcb-a26a-eb0d244710de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:31.028508 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-214c8cd5-bd9c-4fcb-a26a-eb0d244710de service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:31.030403 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da1649-3192-4cbd-9ef3-36ac9628f4ba None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:31.034210 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6b23d-527c-4a8f-b1e4-bac21211e996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:31.034854 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6b23d-527c-4a8f-b1e4-bac21211e996 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:31.035164 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6b23d-527c-4a8f-b1e4-bac21211e996 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:31.035488 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6b23d-527c-4a8f-b1e4-bac21211e996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:31.048756 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6b23d-527c-4a8f-b1e4-bac21211e996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:31.049719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6b23d-527c-4a8f-b1e4-bac21211e996 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:31.060699 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1440/2877] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:00:31.124757 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c8cd5-bd9c-4fcb-a26a-eb0d244710de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:31.126997 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c8cd5-bd9c-4fcb-a26a-eb0d244710de service swift] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:31.137828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1438/2878] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 30 00:00:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 00:00:31.691992 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6b23d-527c-4a8f-b1e4-bac21211e996 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:31.693580 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-74c6b23d-527c-4a8f-b1e4-bac21211e996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:31.708962 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6b23d-527c-4a8f-b1e4-bac21211e996 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:31.714324 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:31.715532 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:31.716077 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:31.716526 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] PATH_INFO: `/v3/projects/c7965c208d7f4436a1f2ff0e6acc5414` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:31.717732 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7965c208d7f4436a1f2ff0e6acc5414)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:31.719768 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:31.727486 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:31.944184 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7965c208d7f4436a1f2ff0e6acc5414', 'request_id': 'req-c268048f-1a10-4a28-93fd-476ffbc2b89d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:31.951902 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7965c208d7f4436a1f2ff0e6acc5414', 'request_id': 'req-c268048f-1a10-4a28-93fd-476ffbc2b89d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:32.048095 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7965c208d7f4436a1f2ff0e6acc5414', 'request_id': 'req-c268048f-1a10-4a28-93fd-476ffbc2b89d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:32.061791 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] The token cache is being invalidate because project c7965c208d7f4436a1f2ff0e6acc5414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:32.062083 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:32.062762 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c268048f-1a10-4a28-93fd-476ffbc2b89d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:32.064298 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1441/2879] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:31 2022] DELETE /identity/v3/projects/c7965c208d7f4436a1f2ff0e6acc5414 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:32.069139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c8cd5-bd9c-4fcb-a26a-eb0d244710de service swift] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:32.081774 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c8cd5-bd9c-4fcb-a26a-eb0d244710de service swift] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:32.160145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c8d9f-1b80-443f-81db-83297a1a2089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:32.161941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c8d9f-1b80-443f-81db-83297a1a2089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:32.162238 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c8d9f-1b80-443f-81db-83297a1a2089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:32.162548 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c8d9f-1b80-443f-81db-83297a1a2089 admin admin] PATH_INFO: `/v3/users/9dd434c375324d3a828fa5265c5bc2c8` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:32.165693 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243a159-99cb-45b1-bf64-765f86465aa7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:32.166033 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243a159-99cb-45b1-bf64-765f86465aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:32.169097 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243a159-99cb-45b1-bf64-765f86465aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:32.169247 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c8d9f-1b80-443f-81db-83297a1a2089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dd434c375324d3a828fa5265c5bc2c8)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:32.169405 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c8d9f-1b80-443f-81db-83297a1a2089 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:32.191349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3243a159-99cb-45b1-bf64-765f86465aa7 None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:32.193044 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3243a159-99cb-45b1-bf64-765f86465aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:32.274137 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1442/2880] 10.222.0.58 () {58 vars in 967 bytes} [Tue Aug 30 00:00:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:32.291520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243a159-99cb-45b1-bf64-765f86465aa7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:32.293820 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3243a159-99cb-45b1-bf64-765f86465aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:32.376907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-310c8d9f-1b80-443f-81db-83297a1a2089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dd434c375324d3a828fa5265c5bc2c8'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:32.380542 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243a159-99cb-45b1-bf64-765f86465aa7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:32.385981 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1e0ca-3d5f-4edd-b3ea-fb941d1b9bc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:32.386825 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e0ca-3d5f-4edd-b3ea-fb941d1b9bc5 service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:32.386965 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e0ca-3d5f-4edd-b3ea-fb941d1b9bc5 service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:32.387311 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e0ca-3d5f-4edd-b3ea-fb941d1b9bc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:32.392334 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-310c8d9f-1b80-443f-81db-83297a1a2089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dd434c375324d3a828fa5265c5bc2c8'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:32.397838 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1e0ca-3d5f-4edd-b3ea-fb941d1b9bc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:32.399558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1e0ca-3d5f-4edd-b3ea-fb941d1b9bc5 service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:32.408454 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-310c8d9f-1b80-443f-81db-83297a1a2089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dd434c375324d3a828fa5265c5bc2c8'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:32.409938 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1443/2881] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 30 00:00:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 00:00:32.494500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-310c8d9f-1b80-443f-81db-83297a1a2089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dd434c375324d3a828fa5265c5bc2c8'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:32.495186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-310c8d9f-1b80-443f-81db-83297a1a2089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dd434c375324d3a828fa5265c5bc2c8'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:32.502023 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1439/2882] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:32 2022] DELETE /identity/v3/users/9dd434c375324d3a828fa5265c5bc2c8 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:00:33.027294 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1e0ca-3d5f-4edd-b3ea-fb941d1b9bc5 service swift] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:33.036201 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-74d1e0ca-3d5f-4edd-b3ea-fb941d1b9bc5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:33.057146 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1e0ca-3d5f-4edd-b3ea-fb941d1b9bc5 service swift] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:33.100683 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:33.147392 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:33.148436 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:33.148851 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:33.149290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] PATH_INFO: `/v3/projects/47078bf69d814e569337b0300a7001aa` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:33.151327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47078bf69d814e569337b0300a7001aa)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:33.155078 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:33.249329 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680abc1-6146-4b15-8c0e-afa26ebaa17e None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:33.249898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680abc1-6146-4b15-8c0e-afa26ebaa17e None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:33.249898 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680abc1-6146-4b15-8c0e-afa26ebaa17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:33.272262 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-7680abc1-6146-4b15-8c0e-afa26ebaa17e None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:33.273627 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7680abc1-6146-4b15-8c0e-afa26ebaa17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:33.278981 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:33.335090 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47078bf69d814e569337b0300a7001aa', 'request_id': 'req-5b65a6dd-354b-4dba-9231-fcc92abd50b6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:33.355262 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47078bf69d814e569337b0300a7001aa', 'request_id': 'req-5b65a6dd-354b-4dba-9231-fcc92abd50b6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:33.376452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1440/2883] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 30 00:00:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:00:33.391855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680abc1-6146-4b15-8c0e-afa26ebaa17e None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:33.393331 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7680abc1-6146-4b15-8c0e-afa26ebaa17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:33.471662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47078bf69d814e569337b0300a7001aa', 'request_id': 'req-5b65a6dd-354b-4dba-9231-fcc92abd50b6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:33.480096 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680abc1-6146-4b15-8c0e-afa26ebaa17e None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:33.492482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] The token cache is being invalidate because project 47078bf69d814e569337b0300a7001aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:33.492834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:33.493787 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5b65a6dd-354b-4dba-9231-fcc92abd50b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:33.496872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1444/2884] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:33 2022] DELETE /identity/v3/projects/47078bf69d814e569337b0300a7001aa => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:33.611417 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6ec77-178d-4512-bae0-32fec013a6dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:33.612522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6ec77-178d-4512-bae0-32fec013a6dd service swift] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:33.613084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6ec77-178d-4512-bae0-32fec013a6dd service swift] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:33.613585 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6ec77-178d-4512-bae0-32fec013a6dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:33.617186 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a9a6ec77-178d-4512-bae0-32fec013a6dd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:33.726527 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6ec77-178d-4512-bae0-32fec013a6dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:33.727786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6ec77-178d-4512-bae0-32fec013a6dd service swift] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:33.737295 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=100 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:33.747367 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1441/2885] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 30 00:00:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 00:00:34.471195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5cb46-c1a7-4d25-95c8-a4a15b2d3eb5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:34.471694 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5cb46-c1a7-4d25-95c8-a4a15b2d3eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:34.471811 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5cb46-c1a7-4d25-95c8-a4a15b2d3eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:34.492753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e2b5cb46-c1a7-4d25-95c8-a4a15b2d3eb5 None None] MFA Rules not processed for user `ff5b4f2f41604a8286141bce23db35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:34.494017 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e2b5cb46-c1a7-4d25-95c8-a4a15b2d3eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:34.502675 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1445/2886] 10.222.0.58 () {58 vars in 967 bytes} [Tue Aug 30 00:00:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:34.515831 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6ec77-178d-4512-bae0-32fec013a6dd service swift] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:34.517444 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-a9a6ec77-178d-4512-bae0-32fec013a6dd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:34.532218 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6ec77-178d-4512-bae0-32fec013a6dd service swift] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:34.540365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ab8ae-6c77-4640-b8ae-3258da104534 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6b37b09a1ae45e8bc16adfa59500eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:34.540996 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ab8ae-6c77-4640-b8ae-3258da104534 service swift] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:34.541276 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ab8ae-6c77-4640-b8ae-3258da104534 service swift] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:34.541551 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ab8ae-6c77-4640-b8ae-3258da104534 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:34.551443 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ab8ae-6c77-4640-b8ae-3258da104534 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:34.553054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ab8ae-6c77-4640-b8ae-3258da104534 service swift] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:34.573074 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1442/2887] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 30 00:00:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:00:37.721163 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5cb46-c1a7-4d25-95c8-a4a15b2d3eb5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:37.723092 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e2b5cb46-c1a7-4d25-95c8-a4a15b2d3eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:37.818376 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5cb46-c1a7-4d25-95c8-a4a15b2d3eb5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:37.826401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:37.827509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:37.827998 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:37.828497 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] PATH_INFO: `/v3/users/42a2a81700ff44c394602c9a5fac5df0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:37.830349 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a2a81700ff44c394602c9a5fac5df0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:37.834239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:38.082623 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a2a81700ff44c394602c9a5fac5df0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:38.089235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:38.097096 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a2a81700ff44c394602c9a5fac5df0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:38.105046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a2a81700ff44c394602c9a5fac5df0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:38.173940 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a2a81700ff44c394602c9a5fac5df0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:38.174696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a2a81700ff44c394602c9a5fac5df0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:38.182812 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1446/2888] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:00:37 2022] DELETE /identity/v3/users/42a2a81700ff44c394602c9a5fac5df0 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:38.188319 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699275ca-7e11-48a3-8cd1-e3931dc9e85f None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:38.188702 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699275ca-7e11-48a3-8cd1-e3931dc9e85f None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:38.188869 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699275ca-7e11-48a3-8cd1-e3931dc9e85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:38.217437 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-699275ca-7e11-48a3-8cd1-e3931dc9e85f None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:00:38.221806 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-699275ca-7e11-48a3-8cd1-e3931dc9e85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:38.307411 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1443/2889] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:00:38.318131 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:38.319612 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:38.380710 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5c4c0-4b34-4266-9bad-897d13f68c12 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:38.387460 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb0f06-2c70-417d-8d44-d49f4c6ea47f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:38.388520 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb0f06-2c70-417d-8d44-d49f4c6ea47f service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:38.388990 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb0f06-2c70-417d-8d44-d49f4c6ea47f service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:38.389525 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb0f06-2c70-417d-8d44-d49f4c6ea47f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:38.401731 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb0f06-2c70-417d-8d44-d49f4c6ea47f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:38.404045 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb0f06-2c70-417d-8d44-d49f4c6ea47f service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:38.414219 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1447/2890] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:00:38.680954 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699275ca-7e11-48a3-8cd1-e3931dc9e85f None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:00:38.684905 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-699275ca-7e11-48a3-8cd1-e3931dc9e85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:00:38.713999 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699275ca-7e11-48a3-8cd1-e3931dc9e85f None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:00:38.721984 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:00:38.723107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:00:38.723684 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:00:38.724037 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] PATH_INFO: `/v3/projects/afc6be07438a47fb9b13974bd61fbe57` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:00:38.725944 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc6be07438a47fb9b13974bd61fbe57)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:00:38.727977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:00:38.867888 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc6be07438a47fb9b13974bd61fbe57', 'request_id': 'req-6115113b-ff59-446e-a64d-eca4260e6b91'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:38.957097 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc6be07438a47fb9b13974bd61fbe57', 'request_id': 'req-6115113b-ff59-446e-a64d-eca4260e6b91'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:38.963835 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc6be07438a47fb9b13974bd61fbe57', 'request_id': 'req-6115113b-ff59-446e-a64d-eca4260e6b91'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:38.977156 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] The token cache is being invalidate because project afc6be07438a47fb9b13974bd61fbe57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:00:38.977445 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:38.978130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:00:38.979663 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1444/2891] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:00:38 2022] DELETE /identity/v3/projects/afc6be07438a47fb9b13974bd61fbe57 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:00:48.409664 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=36,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:00:48.973586 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=61,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:01:05.089232 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb0f06-2c70-417d-8d44-d49f4c6ea47f service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:01:05.090789 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d3eb0f06-2c70-417d-8d44-d49f4c6ea47f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:01:05.104014 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb0f06-2c70-417d-8d44-d49f4c6ea47f service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:01:05.156046 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc422c5-fffc-4411-bc69-5911b57eb2d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:01:05.156758 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc422c5-fffc-4411-bc69-5911b57eb2d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:01:05.157547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc422c5-fffc-4411-bc69-5911b57eb2d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:01:05.157759 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc422c5-fffc-4411-bc69-5911b57eb2d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:01:05.159724 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3bc422c5-fffc-4411-bc69-5911b57eb2d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:01:05.240130 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc422c5-fffc-4411-bc69-5911b57eb2d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:01:05.241567 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc422c5-fffc-4411-bc69-5911b57eb2d1 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:01:05.254841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1448/2892] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:01:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:01:06.443960 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:01:06.445457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:01:06.458471 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115113b-ff59-446e-a64d-eca4260e6b91 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:01:06.462238 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cd20f-a40f-4e0a-b250-f1b63c5aca51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:01:06.463901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd20f-a40f-4e0a-b250-f1b63c5aca51 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:01:06.464377 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd20f-a40f-4e0a-b250-f1b63c5aca51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:01:06.465076 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd20f-a40f-4e0a-b250-f1b63c5aca51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:01:06.467078 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-d76cd20f-a40f-4e0a-b250-f1b63c5aca51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:01:06.484886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cd20f-a40f-4e0a-b250-f1b63c5aca51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:01:06.487083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cd20f-a40f-4e0a-b250-f1b63c5aca51 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:01:06.499279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1445/2893] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:01:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:01:10.001105 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7654a0-7d0e-44ac-8e25-a221f83e00a2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:01:10.002240 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7654a0-7d0e-44ac-8e25-a221f83e00a2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:01:10.002366 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7654a0-7d0e-44ac-8e25-a221f83e00a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:01:10.022067 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-fa7654a0-7d0e-44ac-8e25-a221f83e00a2 None None] MFA Rules not processed for user `1e63436f5691431d802706a8d5a6df7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:01:10.023337 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fa7654a0-7d0e-44ac-8e25-a221f83e00a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:01:10.083288 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1449/2894] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:01:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:01:16.493638 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=6 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:01:20.075639 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=79 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:01:27.062723 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcd154-6a48-4d2f-9721-9e6a82f5549c None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:01:27.063301 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcd154-6a48-4d2f-9721-9e6a82f5549c None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:01:27.063562 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcd154-6a48-4d2f-9721-9e6a82f5549c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:01:27.084054 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-24bcd154-6a48-4d2f-9721-9e6a82f5549c None None] MFA Rules not processed for user `bf8fecbb57394e2e8f5dabfd75ac6b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:01:27.085267 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-24bcd154-6a48-4d2f-9721-9e6a82f5549c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:01:27.147335 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1446/2895] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:01:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:01:28.942248 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7654a0-7d0e-44ac-8e25-a221f83e00a2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:01:28.943735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fa7654a0-7d0e-44ac-8e25-a221f83e00a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:01:29.075123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7654a0-7d0e-44ac-8e25-a221f83e00a2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:01:29.083216 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342e30e-9d9d-449a-ba09-2d028745a8bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:01:29.084170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342e30e-9d9d-449a-ba09-2d028745a8bd service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:01:29.084821 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342e30e-9d9d-449a-ba09-2d028745a8bd service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:01:29.085163 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342e30e-9d9d-449a-ba09-2d028745a8bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:01:29.089705 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4342e30e-9d9d-449a-ba09-2d028745a8bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:01:29.112903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342e30e-9d9d-449a-ba09-2d028745a8bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:01:29.116640 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342e30e-9d9d-449a-ba09-2d028745a8bd service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:01:29.136263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1450/2896] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:01:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:01:37.140382 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=25 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:01:39.127296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=32 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:02:13.369433 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcd154-6a48-4d2f-9721-9e6a82f5549c None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:13.371181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-24bcd154-6a48-4d2f-9721-9e6a82f5549c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:02:13.383901 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcd154-6a48-4d2f-9721-9e6a82f5549c None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:13.387574 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24279173-ace2-41da-9961-1eb5461d1654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:13.388183 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24279173-ace2-41da-9961-1eb5461d1654 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:13.388457 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24279173-ace2-41da-9961-1eb5461d1654 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:13.388736 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24279173-ace2-41da-9961-1eb5461d1654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:13.390513 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-24279173-ace2-41da-9961-1eb5461d1654 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:02:13.401941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24279173-ace2-41da-9961-1eb5461d1654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:13.403416 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24279173-ace2-41da-9961-1eb5461d1654 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:13.412667 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1447/2897] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:02:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:02:23.408280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=6 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:02:38.852334 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342e30e-9d9d-449a-ba09-2d028745a8bd service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:38.854002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4342e30e-9d9d-449a-ba09-2d028745a8bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:02:38.924696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342e30e-9d9d-449a-ba09-2d028745a8bd service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:38.929377 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431f5ac-b390-48e2-87a6-a501dd2019fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:38.930002 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431f5ac-b390-48e2-87a6-a501dd2019fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:38.930281 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431f5ac-b390-48e2-87a6-a501dd2019fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:38.930590 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431f5ac-b390-48e2-87a6-a501dd2019fb admin admin] PATH_INFO: `/v3/users/bf8fecbb57394e2e8f5dabfd75ac6b7c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:38.931714 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431f5ac-b390-48e2-87a6-a501dd2019fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf8fecbb57394e2e8f5dabfd75ac6b7c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:38.934327 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431f5ac-b390-48e2-87a6-a501dd2019fb admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:39.122170 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6431f5ac-b390-48e2-87a6-a501dd2019fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf8fecbb57394e2e8f5dabfd75ac6b7c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:02:39.134824 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6431f5ac-b390-48e2-87a6-a501dd2019fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf8fecbb57394e2e8f5dabfd75ac6b7c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:02:39.143344 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6431f5ac-b390-48e2-87a6-a501dd2019fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf8fecbb57394e2e8f5dabfd75ac6b7c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:02:39.210626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6431f5ac-b390-48e2-87a6-a501dd2019fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf8fecbb57394e2e8f5dabfd75ac6b7c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:02:39.211245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6431f5ac-b390-48e2-87a6-a501dd2019fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf8fecbb57394e2e8f5dabfd75ac6b7c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:02:39.217872 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1451/2898] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:02:38 2022] DELETE /identity/v3/users/bf8fecbb57394e2e8f5dabfd75ac6b7c => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:02:39.490695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24279173-ace2-41da-9961-1eb5461d1654 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:39.503890 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24279173-ace2-41da-9961-1eb5461d1654 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:39.562363 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:39.563018 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:39.563292 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:39.563559 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] PATH_INFO: `/v3/projects/3771a66c80964a40805c3a9f1f51c3da` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:39.564676 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3771a66c80964a40805c3a9f1f51c3da)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:39.566590 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:39.667750 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3771a66c80964a40805c3a9f1f51c3da', 'request_id': 'req-37d29960-dca3-4f90-aac8-675aebf52247'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:02:39.740343 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3771a66c80964a40805c3a9f1f51c3da', 'request_id': 'req-37d29960-dca3-4f90-aac8-675aebf52247'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:02:39.746787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3771a66c80964a40805c3a9f1f51c3da', 'request_id': 'req-37d29960-dca3-4f90-aac8-675aebf52247'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:02:39.757920 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] The token cache is being invalidate because project 3771a66c80964a40805c3a9f1f51c3da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81853) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:02:39.758206 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:02:39.758860 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-37d29960-dca3-4f90-aac8-675aebf52247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:02:39.760621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1448/2899] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:02:39 2022] DELETE /identity/v3/projects/3771a66c80964a40805c3a9f1f51c3da => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:02:39.765899 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9232fa-7dcf-4be4-a278-adcd6538dc5c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:39.766208 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9232fa-7dcf-4be4-a278-adcd6538dc5c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:39.766479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9232fa-7dcf-4be4-a278-adcd6538dc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:39.787213 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2c9232fa-7dcf-4be4-a278-adcd6538dc5c None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:02:39.788402 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2c9232fa-7dcf-4be4-a278-adcd6538dc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:02:39.846614 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1452/2900] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:02:39.855880 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685553f6-551f-4083-99ee-2b65ab1bb986 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:39.856175 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685553f6-551f-4083-99ee-2b65ab1bb986 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:39.856442 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685553f6-551f-4083-99ee-2b65ab1bb986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:39.876046 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-685553f6-551f-4083-99ee-2b65ab1bb986 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:02:39.877257 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-685553f6-551f-4083-99ee-2b65ab1bb986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:02:39.885120 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1449/2901] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:02:39.889777 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9232fa-7dcf-4be4-a278-adcd6538dc5c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:39.899421 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9232fa-7dcf-4be4-a278-adcd6538dc5c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:39.904572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75075156-a8c1-4087-bf69-5d3ea784de15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:39.905168 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075156-a8c1-4087-bf69-5d3ea784de15 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:39.905447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075156-a8c1-4087-bf69-5d3ea784de15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:39.905739 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075156-a8c1-4087-bf69-5d3ea784de15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:39.906356 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075156-a8c1-4087-bf69-5d3ea784de15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:02:39.906696 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075156-a8c1-4087-bf69-5d3ea784de15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:39.908301 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075156-a8c1-4087-bf69-5d3ea784de15 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:39.920461 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1453/2902] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 30 00:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:02:39.924867 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685553f6-551f-4083-99ee-2b65ab1bb986 None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:39.934252 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685553f6-551f-4083-99ee-2b65ab1bb986 None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:39.939032 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a773693-175f-48a1-83a5-891f295dd1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:39.939649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a773693-175f-48a1-83a5-891f295dd1cb admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:39.939927 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a773693-175f-48a1-83a5-891f295dd1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:39.940195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a773693-175f-48a1-83a5-891f295dd1cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:39.940821 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a773693-175f-48a1-83a5-891f295dd1cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:39.942761 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a773693-175f-48a1-83a5-891f295dd1cb admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:40.041600 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1450/2903] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 30 00:02:39 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:02:40.046470 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75075156-a8c1-4087-bf69-5d3ea784de15 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:40.059692 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75075156-a8c1-4087-bf69-5d3ea784de15 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:40.064087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:02:40.118248 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7556b-cf10-454a-9e63-3f75d2b8db2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:40.118890 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7556b-cf10-454a-9e63-3f75d2b8db2a admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:40.119212 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7556b-cf10-454a-9e63-3f75d2b8db2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:40.119496 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7556b-cf10-454a-9e63-3f75d2b8db2a admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:40.120748 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7556b-cf10-454a-9e63-3f75d2b8db2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:40.122723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7556b-cf10-454a-9e63-3f75d2b8db2a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:40.193929 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1454/2904] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 30 00:02:40 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:02:40.198509 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a773693-175f-48a1-83a5-891f295dd1cb admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:40.209941 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a773693-175f-48a1-83a5-891f295dd1cb admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:40.215130 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47c356-25ae-47b7-9810-d9d3da202418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:40.215748 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47c356-25ae-47b7-9810-d9d3da202418 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:40.216159 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47c356-25ae-47b7-9810-d9d3da202418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:40.216495 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47c356-25ae-47b7-9810-d9d3da202418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:40.217079 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47c356-25ae-47b7-9810-d9d3da202418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:40.218720 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47c356-25ae-47b7-9810-d9d3da202418 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:40.229855 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1451/2905] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:02:40.233942 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7556b-cf10-454a-9e63-3f75d2b8db2a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:40.244228 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7556b-cf10-454a-9e63-3f75d2b8db2a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:40.250556 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a39d2a-05e2-422b-9b2d-5c3ab5a98707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:40.253551 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a39d2a-05e2-422b-9b2d-5c3ab5a98707 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:40.253551 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a39d2a-05e2-422b-9b2d-5c3ab5a98707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:40.253551 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a39d2a-05e2-422b-9b2d-5c3ab5a98707 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:40.253551 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a39d2a-05e2-422b-9b2d-5c3ab5a98707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:40.254119 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a39d2a-05e2-422b-9b2d-5c3ab5a98707 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:40.265272 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1455/2906] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:02:40.269432 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47c356-25ae-47b7-9810-d9d3da202418 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:40.279531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47c356-25ae-47b7-9810-d9d3da202418 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:40.284275 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac9bde-3acc-44ce-a860-7ed73cb94fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:40.284917 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac9bde-3acc-44ce-a860-7ed73cb94fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:40.285196 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac9bde-3acc-44ce-a860-7ed73cb94fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:40.285468 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac9bde-3acc-44ce-a860-7ed73cb94fdd admin admin] PATH_INFO: `/v3/projects/6516b21ff0b6423cbed032257af13839/users/6825bc701a46434d8bc9dd62c203ec83/roles/4d98a2b16f8f42bca3e5dac0374cdb54` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:40.301200 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac9bde-3acc-44ce-a860-7ed73cb94fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6516b21ff0b6423cbed032257af13839, user_id=6825bc701a46434d8bc9dd62c203ec83, role_id=4d98a2b16f8f42bca3e5dac0374cdb54)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:40.303907 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac9bde-3acc-44ce-a860-7ed73cb94fdd admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:40.399762 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1452/2907] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:02:40 2022] PUT /identity/v3/projects/6516b21ff0b6423cbed032257af13839/users/6825bc701a46434d8bc9dd62c203ec83/roles/4d98a2b16f8f42bca3e5dac0374cdb54 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:02:40.405445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208c14-3cd8-4ee9-a7cb-adfef9451be9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:40.408343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208c14-3cd8-4ee9-a7cb-adfef9451be9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:40.408343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208c14-3cd8-4ee9-a7cb-adfef9451be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:40.425964 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-59208c14-3cd8-4ee9-a7cb-adfef9451be9 None None] MFA Rules not processed for user `557208629c7f4826b594ef28cabef8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:02:40.427192 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-59208c14-3cd8-4ee9-a7cb-adfef9451be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:02:40.486205 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1456/2908] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:02:40.496681 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac9bde-3acc-44ce-a860-7ed73cb94fdd admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:40.498355 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-80ac9bde-3acc-44ce-a860-7ed73cb94fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:02:40.518893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:02:40.554881 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac9bde-3acc-44ce-a860-7ed73cb94fdd admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:40.559195 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6592fa-0f90-447b-92a0-9e42c1ccedd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:40.559828 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6592fa-0f90-447b-92a0-9e42c1ccedd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:40.560107 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6592fa-0f90-447b-92a0-9e42c1ccedd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:40.560379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6592fa-0f90-447b-92a0-9e42c1ccedd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:40.569708 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6592fa-0f90-447b-92a0-9e42c1ccedd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:40.571158 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6592fa-0f90-447b-92a0-9e42c1ccedd9 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:40.581398 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1453/2909] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:02:45.961087 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db364ec4-654a-409a-8844-6e55a4fa1a4b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:45.961576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db364ec4-654a-409a-8844-6e55a4fa1a4b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:45.962150 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db364ec4-654a-409a-8844-6e55a4fa1a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:45.999242 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-db364ec4-654a-409a-8844-6e55a4fa1a4b None None] MFA Rules not processed for user `6825bc701a46434d8bc9dd62c203ec83`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:02:46.003353 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-db364ec4-654a-409a-8844-6e55a4fa1a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:02:46.091802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1457/2910] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:02:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:02:46.097469 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6592fa-0f90-447b-92a0-9e42c1ccedd9 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:46.113621 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6592fa-0f90-447b-92a0-9e42c1ccedd9 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:46.119745 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d48aa-099c-4a21-b444-f3190ba62175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:46.120359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d48aa-099c-4a21-b444-f3190ba62175 admin admin] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:46.120635 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d48aa-099c-4a21-b444-f3190ba62175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:46.120904 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d48aa-099c-4a21-b444-f3190ba62175 admin admin] PATH_INFO: `/v3/projects` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:46.121553 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d48aa-099c-4a21-b444-f3190ba62175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:46.124383 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d48aa-099c-4a21-b444-f3190ba62175 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:46.224293 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1454/2911] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 30 00:02:46 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:02:46.228479 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db364ec4-654a-409a-8844-6e55a4fa1a4b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:46.241316 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db364ec4-654a-409a-8844-6e55a4fa1a4b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:46.262385 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:02:46.313713 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b008c-2eb7-4f8b-92ba-6a2591df79e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:46.314680 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b008c-2eb7-4f8b-92ba-6a2591df79e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:46.315914 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b008c-2eb7-4f8b-92ba-6a2591df79e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:46.316354 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b008c-2eb7-4f8b-92ba-6a2591df79e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:46.317123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b008c-2eb7-4f8b-92ba-6a2591df79e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:46.322223 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b008c-2eb7-4f8b-92ba-6a2591df79e3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:46.407559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1458/2912] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 30 00:02:46 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:02:46.412214 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d48aa-099c-4a21-b444-f3190ba62175 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:46.423952 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d48aa-099c-4a21-b444-f3190ba62175 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:46.428786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6886a-aba0-43f6-ac9f-20d153629a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:46.429534 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6886a-aba0-43f6-ac9f-20d153629a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:46.429815 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6886a-aba0-43f6-ac9f-20d153629a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:46.430087 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6886a-aba0-43f6-ac9f-20d153629a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:46.430827 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6886a-aba0-43f6-ac9f-20d153629a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:46.432439 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6886a-aba0-43f6-ac9f-20d153629a0b admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:46.448289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1455/2913] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:02:46.450457 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b008c-2eb7-4f8b-92ba-6a2591df79e3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:46.463437 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b008c-2eb7-4f8b-92ba-6a2591df79e3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:46.468970 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fd6a2-00b4-41e6-b895-49e90d09adf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:46.470125 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd6a2-00b4-41e6-b895-49e90d09adf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:46.470125 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd6a2-00b4-41e6-b895-49e90d09adf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:46.470389 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd6a2-00b4-41e6-b895-49e90d09adf8 admin admin] PATH_INFO: `/v3/projects/b9da58c01b6a48649611ebb67da31425/users/96cb72de833a4ef196a64f7b3ac157a4/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:46.486830 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fd6a2-00b4-41e6-b895-49e90d09adf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9da58c01b6a48649611ebb67da31425, user_id=96cb72de833a4ef196a64f7b3ac157a4, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:46.489859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fd6a2-00b4-41e6-b895-49e90d09adf8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:46.599328 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1459/2914] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:02:46 2022] PUT /identity/v3/projects/b9da58c01b6a48649611ebb67da31425/users/96cb72de833a4ef196a64f7b3ac157a4/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:02:46.603554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6886a-aba0-43f6-ac9f-20d153629a0b admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:46.614911 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6886a-aba0-43f6-ac9f-20d153629a0b admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:46.676783 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e4ca4-cea3-434f-99ea-e22ac7796e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:46.677413 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e4ca4-cea3-434f-99ea-e22ac7796e85 admin admin] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:46.677674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e4ca4-cea3-434f-99ea-e22ac7796e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:46.677967 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e4ca4-cea3-434f-99ea-e22ac7796e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:46.678646 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e4ca4-cea3-434f-99ea-e22ac7796e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:46.680296 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e4ca4-cea3-434f-99ea-e22ac7796e85 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:46.691099 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1456/2915] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 30 00:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:02:46.695445 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fd6a2-00b4-41e6-b895-49e90d09adf8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:46.707374 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fd6a2-00b4-41e6-b895-49e90d09adf8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:46.712196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744d25d-0326-4146-89f8-bff81e1d7148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:46.712943 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744d25d-0326-4146-89f8-bff81e1d7148 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:46.713226 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744d25d-0326-4146-89f8-bff81e1d7148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:46.713502 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744d25d-0326-4146-89f8-bff81e1d7148 admin admin] PATH_INFO: `/v3/projects/b9da58c01b6a48649611ebb67da31425/users/96cb72de833a4ef196a64f7b3ac157a4/roles/d86dba8efe20423abdde06031690beb8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:46.715506 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744d25d-0326-4146-89f8-bff81e1d7148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9da58c01b6a48649611ebb67da31425, user_id=96cb72de833a4ef196a64f7b3ac157a4, role_id=d86dba8efe20423abdde06031690beb8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:46.719027 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744d25d-0326-4146-89f8-bff81e1d7148 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:46.731100 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1460/2916] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 30 00:02:46 2022] PUT /identity/v3/projects/b9da58c01b6a48649611ebb67da31425/users/96cb72de833a4ef196a64f7b3ac157a4/roles/d86dba8efe20423abdde06031690beb8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:02:52.587121 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab17e5-1290-4e97-b1fa-b7d2e902ec12 None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:52.587554 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab17e5-1290-4e97-b1fa-b7d2e902ec12 None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:52.587639 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab17e5-1290-4e97-b1fa-b7d2e902ec12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:52.622220 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-ddab17e5-1290-4e97-b1fa-b7d2e902ec12 None None] MFA Rules not processed for user `96cb72de833a4ef196a64f7b3ac157a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:02:52.624081 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-ddab17e5-1290-4e97-b1fa-b7d2e902ec12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:02:52.707410 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1457/2917] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:02:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:02:52.725235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744d25d-0326-4146-89f8-bff81e1d7148 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:02:52.727676 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6744d25d-0326-4146-89f8-bff81e1d7148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:02:52.816566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744d25d-0326-4146-89f8-bff81e1d7148 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:02:52.822128 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead7f84-d043-43b3-8872-3cafbedf4384 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:02:52.822772 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead7f84-d043-43b3-8872-3cafbedf4384 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:52.823049 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead7f84-d043-43b3-8872-3cafbedf4384 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:52.823319 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead7f84-d043-43b3-8872-3cafbedf4384 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:52.882763 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead7f84-d043-43b3-8872-3cafbedf4384 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:02:52.884285 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead7f84-d043-43b3-8872-3cafbedf4384 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:02:52.894541 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1461/2918] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:02:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:02:55.601500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4e34d-2a5f-4096-9a80-5364a3c20a9b None None] REQUEST_METHOD: `POST` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:02:55.602045 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4e34d-2a5f-4096-9a80-5364a3c20a9b None None] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:02:55.602163 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4e34d-2a5f-4096-9a80-5364a3c20a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:02:55.623321 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.auth.core [None req-b7e4e34d-2a5f-4096-9a80-5364a3c20a9b None None] MFA Rules not processed for user `6825bc701a46434d8bc9dd62c203ec83`. Rule list: `[]` (Enabled: `True`). {{(pid=81853) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:02:55.624514 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b7e4e34d-2a5f-4096-9a80-5364a3c20a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:02:55.631787 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1458/2919] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:02:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:03:02.889548 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:03:05.616886 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:03:13.790735 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead7f84-d043-43b3-8872-3cafbedf4384 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:03:13.847096 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead7f84-d043-43b3-8872-3cafbedf4384 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:03:13.850753 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167d894-ba96-494b-86df-fd3d30a13926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:03:13.851348 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167d894-ba96-494b-86df-fd3d30a13926 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:03:13.851621 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167d894-ba96-494b-86df-fd3d30a13926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:03:13.851886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167d894-ba96-494b-86df-fd3d30a13926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:03:13.853662 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d167d894-ba96-494b-86df-fd3d30a13926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:03:13.864418 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167d894-ba96-494b-86df-fd3d30a13926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:03:13.865833 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167d894-ba96-494b-86df-fd3d30a13926 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:03:13.873802 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1462/2920] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:03:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:03:17.090415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4e34d-2a5f-4096-9a80-5364a3c20a9b None None] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:03:17.092415 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b7e4e34d-2a5f-4096-9a80-5364a3c20a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:03:17.182626 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4e34d-2a5f-4096-9a80-5364a3c20a9b None None] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:03:17.188579 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b044473-a6d4-4dfc-a526-34283f83dc42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64a42591a6814a0cbcb8444829cb25b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:03:17.189563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b044473-a6d4-4dfc-a526-34283f83dc42 service cinder] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:03:17.189753 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b044473-a6d4-4dfc-a526-34283f83dc42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:03:17.190950 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b044473-a6d4-4dfc-a526-34283f83dc42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:03:17.192864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-4b044473-a6d4-4dfc-a526-34283f83dc42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:03:17.263529 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b044473-a6d4-4dfc-a526-34283f83dc42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:03:17.265729 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b044473-a6d4-4dfc-a526-34283f83dc42 service cinder] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:03:17.276602 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1459/2921] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:03:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:03:23.870109 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=26 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:03:27.075901 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77392ba-27e3-4baa-879d-fef32d714610 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:03:27.076601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77392ba-27e3-4baa-879d-fef32d714610 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:03:27.076601 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77392ba-27e3-4baa-879d-fef32d714610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:03:27.097494 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d77392ba-27e3-4baa-879d-fef32d714610 None None] MFA Rules not processed for user `1e63436f5691431d802706a8d5a6df7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:03:27.098754 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d77392ba-27e3-4baa-879d-fef32d714610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:03:27.105723 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1463/2922] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 30 00:03:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:03:27.271698 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=48 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:03:27.468947 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b044473-a6d4-4dfc-a526-34283f83dc42 service cinder] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:03:27.479482 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b044473-a6d4-4dfc-a526-34283f83dc42 service cinder] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:03:27.483379 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f4ffe-120c-43d0-8c5b-33633a143dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:03:27.483979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f4ffe-120c-43d0-8c5b-33633a143dfe service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:03:27.484395 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f4ffe-120c-43d0-8c5b-33633a143dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:03:27.484674 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f4ffe-120c-43d0-8c5b-33633a143dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:03:27.486772 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-cf3f4ffe-120c-43d0-8c5b-33633a143dfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:03:27.497147 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f4ffe-120c-43d0-8c5b-33633a143dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:03:27.498649 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f4ffe-120c-43d0-8c5b-33633a143dfe service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:03:27.508136 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1460/2923] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:03:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:03:32.338237 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77392ba-27e3-4baa-879d-fef32d714610 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:03:32.339859 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d77392ba-27e3-4baa-879d-fef32d714610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:03:32.364469 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77392ba-27e3-4baa-879d-fef32d714610 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:03:32.371241 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d7ac-9f07-4eb2-91c7-5840b5cb9c5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:03:32.371866 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d7ac-9f07-4eb2-91c7-5840b5cb9c5a service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:03:32.372144 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d7ac-9f07-4eb2-91c7-5840b5cb9c5a service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:03:32.372419 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d7ac-9f07-4eb2-91c7-5840b5cb9c5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:03:32.374357 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e0c8d7ac-9f07-4eb2-91c7-5840b5cb9c5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:03:32.387946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8d7ac-9f07-4eb2-91c7-5840b5cb9c5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:03:32.389566 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8d7ac-9f07-4eb2-91c7-5840b5cb9c5a service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:03:32.402946 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1464/2924] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:03:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:03:34.900365 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f4ffe-120c-43d0-8c5b-33633a143dfe service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:03:34.901843 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-cf3f4ffe-120c-43d0-8c5b-33633a143dfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:03:34.912271 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f4ffe-120c-43d0-8c5b-33633a143dfe service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:03:34.915851 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795285ca-197b-4b6c-9243-e19b88c43639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:03:34.916436 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795285ca-197b-4b6c-9243-e19b88c43639 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:03:34.916714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795285ca-197b-4b6c-9243-e19b88c43639 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:03:34.916982 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795285ca-197b-4b6c-9243-e19b88c43639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:03:34.926327 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795285ca-197b-4b6c-9243-e19b88c43639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:03:34.927779 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795285ca-197b-4b6c-9243-e19b88c43639 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:03:34.938281 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1461/2925] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:03:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:03:42.397993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=9 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:03:44.933868 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=12 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:03:49.615932 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d7ac-9f07-4eb2-91c7-5840b5cb9c5a service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:03:49.627950 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d7ac-9f07-4eb2-91c7-5840b5cb9c5a service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:03:49.634745 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96ea76-dff6-4293-a192-0a255345bb44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:03:49.635492 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96ea76-dff6-4293-a192-0a255345bb44 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:03:49.635776 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96ea76-dff6-4293-a192-0a255345bb44 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:03:49.636172 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96ea76-dff6-4293-a192-0a255345bb44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:03:49.637947 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0f96ea76-dff6-4293-a192-0a255345bb44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:03:49.651051 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96ea76-dff6-4293-a192-0a255345bb44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:03:49.652701 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96ea76-dff6-4293-a192-0a255345bb44 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:03:49.664101 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1465/2926] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:03:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:03:59.660251 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:05:13.813151 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795285ca-197b-4b6c-9243-e19b88c43639 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:05:13.814605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-795285ca-197b-4b6c-9243-e19b88c43639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:05:13.883711 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795285ca-197b-4b6c-9243-e19b88c43639 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:05:13.889393 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f4879-e548-40b5-a55a-d035de3d0b6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:05:13.889990 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4879-e548-40b5-a55a-d035de3d0b6a service placement] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:05:13.890264 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4879-e548-40b5-a55a-d035de3d0b6a service placement] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:05:13.890584 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4879-e548-40b5-a55a-d035de3d0b6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:05:13.892304 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-9c4f4879-e548-40b5-a55a-d035de3d0b6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:05:13.904182 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f4879-e548-40b5-a55a-d035de3d0b6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:05:13.905735 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f4879-e548-40b5-a55a-d035de3d0b6a service placement] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:05:13.915523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1462/2927] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:05:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:05:23.911280 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=32 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:05:38.171963 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96ea76-dff6-4293-a192-0a255345bb44 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:05:38.185410 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96ea76-dff6-4293-a192-0a255345bb44 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:05:38.192152 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9a40-69ef-49fa-97be-8edc1af6ffb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a8261d92064b838fc6e8f415879566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:05:38.193001 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9a40-69ef-49fa-97be-8edc1af6ffb7 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:05:38.193447 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9a40-69ef-49fa-97be-8edc1af6ffb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:05:38.194652 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9a40-69ef-49fa-97be-8edc1af6ffb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:05:38.198644 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-aa6e9a40-69ef-49fa-97be-8edc1af6ffb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:05:38.214401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e9a40-69ef-49fa-97be-8edc1af6ffb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:05:38.216605 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e9a40-69ef-49fa-97be-8edc1af6ffb7 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:05:38.230234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1466/2928] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:05:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:05:44.664354 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f4879-e548-40b5-a55a-d035de3d0b6a service placement] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:05:44.674786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f4879-e548-40b5-a55a-d035de3d0b6a service placement] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:05:44.678399 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da303711-ba5f-4502-8c8c-819a377f4764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:05:44.679021 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303711-ba5f-4502-8c8c-819a377f4764 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:05:44.679294 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303711-ba5f-4502-8c8c-819a377f4764 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:05:44.679563 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da303711-ba5f-4502-8c8c-819a377f4764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:05:44.681345 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-da303711-ba5f-4502-8c8c-819a377f4764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:05:44.744044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da303711-ba5f-4502-8c8c-819a377f4764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:05:44.745463 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da303711-ba5f-4502-8c8c-819a377f4764 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:05:44.756523 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1463/2929] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:05:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:05:48.225788 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:05:48.368500 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9a40-69ef-49fa-97be-8edc1af6ffb7 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:05:48.380588 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9a40-69ef-49fa-97be-8edc1af6ffb7 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:05:48.385999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eb71d-4839-47a1-b09f-763de8ae53d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:05:48.386743 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eb71d-4839-47a1-b09f-763de8ae53d3 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:05:48.387126 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eb71d-4839-47a1-b09f-763de8ae53d3 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:05:48.387519 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eb71d-4839-47a1-b09f-763de8ae53d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:05:48.389490 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a57eb71d-4839-47a1-b09f-763de8ae53d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:05:48.403245 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57eb71d-4839-47a1-b09f-763de8ae53d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:05:48.405966 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57eb71d-4839-47a1-b09f-763de8ae53d3 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:05:48.419603 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1467/2930] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:05:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:05:51.202840 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da303711-ba5f-4502-8c8c-819a377f4764 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:05:51.210910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da303711-ba5f-4502-8c8c-819a377f4764 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:05:51.214926 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5209c2d-538c-4b4f-922e-3782f29505e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:05:51.215528 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5209c2d-538c-4b4f-922e-3782f29505e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:05:51.215811 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5209c2d-538c-4b4f-922e-3782f29505e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:05:51.216083 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5209c2d-538c-4b4f-922e-3782f29505e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:05:51.217826 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f5209c2d-538c-4b4f-922e-3782f29505e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:05:51.227979 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5209c2d-538c-4b4f-922e-3782f29505e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:05:51.229516 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5209c2d-538c-4b4f-922e-3782f29505e8 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:05:51.242668 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1464/2931] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:05:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:05:52.326559 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eb71d-4839-47a1-b09f-763de8ae53d3 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:05:52.346487 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eb71d-4839-47a1-b09f-763de8ae53d3 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:05:52.354355 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fcc1d-e253-44c3-bfff-71e6d2d67f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:05:52.355200 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fcc1d-e253-44c3-bfff-71e6d2d67f65 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:05:52.355547 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fcc1d-e253-44c3-bfff-71e6d2d67f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:05:52.355886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fcc1d-e253-44c3-bfff-71e6d2d67f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:05:52.359620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-839fcc1d-e253-44c3-bfff-71e6d2d67f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:05:52.397504 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839fcc1d-e253-44c3-bfff-71e6d2d67f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:05:52.399406 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839fcc1d-e253-44c3-bfff-71e6d2d67f65 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:05:52.412903 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1468/2932] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:05:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:05:52.820524 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5209c2d-538c-4b4f-922e-3782f29505e8 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:05:52.821939 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-f5209c2d-538c-4b4f-922e-3782f29505e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:05:52.837394 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5209c2d-538c-4b4f-922e-3782f29505e8 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:05:52.842024 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:05:52.842786 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:05:52.843070 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:05:52.843369 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] PATH_INFO: `/v3/users/1e63436f5691431d802706a8d5a6df7f` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:05:52.844522 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e63436f5691431d802706a8d5a6df7f)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:05:52.846519 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:05:53.036139 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e63436f5691431d802706a8d5a6df7f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:53.049447 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e63436f5691431d802706a8d5a6df7f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:53.056289 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e63436f5691431d802706a8d5a6df7f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:53.133337 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e63436f5691431d802706a8d5a6df7f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:53.133971 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e63436f5691431d802706a8d5a6df7f'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:53.140520 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1465/2933] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:05:52 2022] DELETE /identity/v3/users/1e63436f5691431d802706a8d5a6df7f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:05:53.412999 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fcc1d-e253-44c3-bfff-71e6d2d67f65 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:05:53.425712 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fcc1d-e253-44c3-bfff-71e6d2d67f65 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:05:53.484233 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:05:53.484870 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:05:53.485877 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:05:53.486234 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] PATH_INFO: `/v3/projects/c78ff4cbc0994af8855b1ac67cb281fe` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:05:53.487576 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78ff4cbc0994af8855b1ac67cb281fe)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:05:53.489808 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:05:53.593597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78ff4cbc0994af8855b1ac67cb281fe', 'request_id': 'req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:53.602900 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78ff4cbc0994af8855b1ac67cb281fe', 'request_id': 'req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:53.665570 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78ff4cbc0994af8855b1ac67cb281fe', 'request_id': 'req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:53.680211 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] The token cache is being invalidate because project c78ff4cbc0994af8855b1ac67cb281fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:05:53.680489 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:53.681149 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:53.682671 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1469/2934] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:05:53 2022] DELETE /identity/v3/projects/c78ff4cbc0994af8855b1ac67cb281fe => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:05:53.686726 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:05:53.689168 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:05:53.702366 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b3586-1260-4196-9a85-710fafde8762 admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:05:53.757564 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:05:53.769302 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:05:53.770069 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:05:53.770500 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:05:53.771145 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] PATH_INFO: `/v3/users/8ea434be0ec14904809ba7b48426a372` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:05:53.772531 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ea434be0ec14904809ba7b48426a372)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:05:53.774547 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:05:54.036084 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ea434be0ec14904809ba7b48426a372'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:54.048672 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ea434be0ec14904809ba7b48426a372'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:54.055565 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ea434be0ec14904809ba7b48426a372'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:54.135958 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ea434be0ec14904809ba7b48426a372'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:54.136620 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ea434be0ec14904809ba7b48426a372'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:54.143973 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1466/2935] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:05:53 2022] DELETE /identity/v3/users/8ea434be0ec14904809ba7b48426a372 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:05:54.399809 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:05:54.401291 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:05:54.417158 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9c765-5fdc-46b7-8d1f-e3a7af5f5980 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:05:54.475689 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:05:54.476317 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:05:54.476595 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:05:54.476869 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] PATH_INFO: `/v3/projects/6a86a9923ac94e39b93c5f2107a1f3cf` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:05:54.477983 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a86a9923ac94e39b93c5f2107a1f3cf)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:05:54.479926 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:05:54.486597 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:05:54.585422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a86a9923ac94e39b93c5f2107a1f3cf', 'request_id': 'req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:54.590868 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a86a9923ac94e39b93c5f2107a1f3cf', 'request_id': 'req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:54.702161 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a86a9923ac94e39b93c5f2107a1f3cf', 'request_id': 'req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:54.715076 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] The token cache is being invalidate because project 6a86a9923ac94e39b93c5f2107a1f3cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:05:54.715435 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:54.716371 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:05:54.718268 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1470/2936] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:05:54 2022] DELETE /identity/v3/projects/6a86a9923ac94e39b93c5f2107a1f3cf => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:06:04.139452 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:06:04.717343 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:06:06.658797 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:06:06.660279 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:06:06.716903 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58384ce-f4ca-49fc-b56a-876c14bee10c admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:06:06.769539 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762bd44-400d-499c-8c1b-601d7c7aaa36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:06:06.770140 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762bd44-400d-499c-8c1b-601d7c7aaa36 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:06:06.770441 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762bd44-400d-499c-8c1b-601d7c7aaa36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:06:06.770714 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762bd44-400d-499c-8c1b-601d7c7aaa36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:06:06.772453 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3762bd44-400d-499c-8c1b-601d7c7aaa36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:06:06.847478 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762bd44-400d-499c-8c1b-601d7c7aaa36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:06:06.848989 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762bd44-400d-499c-8c1b-601d7c7aaa36 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:06:06.861255 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1467/2937] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:06:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:06:08.048921 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:06:08.050408 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:06:08.062911 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc079f5-71b6-46c8-8ac5-7ff44da19da5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:06:08.067536 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ef4eb-69d0-4392-b69f-67715edab46e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:06:08.068143 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ef4eb-69d0-4392-b69f-67715edab46e service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:06:08.068422 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ef4eb-69d0-4392-b69f-67715edab46e service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:06:08.068719 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ef4eb-69d0-4392-b69f-67715edab46e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:06:08.070493 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b22ef4eb-69d0-4392-b69f-67715edab46e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:06:08.085690 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ef4eb-69d0-4392-b69f-67715edab46e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:06:08.087263 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ef4eb-69d0-4392-b69f-67715edab46e service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:06:08.099394 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1471/2938] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:06:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:06:16.855317 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=74 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:06:18.094025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:07:20.453142 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762bd44-400d-499c-8c1b-601d7c7aaa36 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:07:20.454673 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-3762bd44-400d-499c-8c1b-601d7c7aaa36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:07:20.466868 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762bd44-400d-499c-8c1b-601d7c7aaa36 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:07:20.470864 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02861a81-3cfc-4aac-ae01-4853995f2cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:07:20.471504 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02861a81-3cfc-4aac-ae01-4853995f2cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:07:20.472312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02861a81-3cfc-4aac-ae01-4853995f2cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:07:20.472893 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02861a81-3cfc-4aac-ae01-4853995f2cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:07:20.474910 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-02861a81-3cfc-4aac-ae01-4853995f2cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:07:20.485882 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02861a81-3cfc-4aac-ae01-4853995f2cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:07:20.487965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02861a81-3cfc-4aac-ae01-4853995f2cc6 service neutron] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:07:20.497050 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1468/2939] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:07:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:07:30.493359 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=6 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:07:41.907482 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ef4eb-69d0-4392-b69f-67715edab46e service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:07:41.917841 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ef4eb-69d0-4392-b69f-67715edab46e service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:07:41.921599 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b4590-fb04-4eca-8fbf-98da25296e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '863bcfa6b8694182b9d163133a141907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:07:41.922195 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4590-fb04-4eca-8fbf-98da25296e3f service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:07:41.922508 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4590-fb04-4eca-8fbf-98da25296e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:07:41.925664 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4590-fb04-4eca-8fbf-98da25296e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:07:41.925664 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-141b4590-fb04-4eca-8fbf-98da25296e3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:07:42.025296 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b4590-fb04-4eca-8fbf-98da25296e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:07:42.027196 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b4590-fb04-4eca-8fbf-98da25296e3f service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:07:42.047037 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1472/2940] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:07:44.106977 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02861a81-3cfc-4aac-ae01-4853995f2cc6 service neutron] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:07:44.108484 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-02861a81-3cfc-4aac-ae01-4853995f2cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:07:44.123448 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02861a81-3cfc-4aac-ae01-4853995f2cc6 service neutron] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:07:44.127916 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:07:44.128899 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:07:44.129312 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:07:44.129583 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] PATH_INFO: `/v3/users/6825bc701a46434d8bc9dd62c203ec83` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:07:44.130741 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6825bc701a46434d8bc9dd62c203ec83)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:07:44.132655 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:07:44.301619 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6825bc701a46434d8bc9dd62c203ec83'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:44.313633 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6825bc701a46434d8bc9dd62c203ec83'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:44.320936 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6825bc701a46434d8bc9dd62c203ec83'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:44.390543 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6825bc701a46434d8bc9dd62c203ec83'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:44.391091 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6825bc701a46434d8bc9dd62c203ec83'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:44.398181 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1469/2941] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:07:44 2022] DELETE /identity/v3/users/6825bc701a46434d8bc9dd62c203ec83 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:07:44.655558 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b4590-fb04-4eca-8fbf-98da25296e3f service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:07:44.667104 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b4590-fb04-4eca-8fbf-98da25296e3f service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:07:44.724289 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:07:44.724951 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:07:44.725235 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:07:44.725509 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] PATH_INFO: `/v3/projects/6516b21ff0b6423cbed032257af13839` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:07:44.726658 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6516b21ff0b6423cbed032257af13839)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:07:44.728572 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:07:44.820610 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6516b21ff0b6423cbed032257af13839', 'request_id': 'req-aa60cc43-2955-48b9-b3de-4a10eaf80b26'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:44.826123 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6516b21ff0b6423cbed032257af13839', 'request_id': 'req-aa60cc43-2955-48b9-b3de-4a10eaf80b26'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:44.885025 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6516b21ff0b6423cbed032257af13839', 'request_id': 'req-aa60cc43-2955-48b9-b3de-4a10eaf80b26'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:44.899401 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] The token cache is being invalidate because project 6516b21ff0b6423cbed032257af13839 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:07:44.899626 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:44.900290 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:44.901782 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1473/2942] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:07:44 2022] DELETE /identity/v3/projects/6516b21ff0b6423cbed032257af13839 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:07:44.905965 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:07:44.907521 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:07:44.921396 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a36643-500f-4300-a9b6-f3c6f3d3c81e admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:07:44.978349 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:07:44.980943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:07:44.980943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:07:44.980943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] PATH_INFO: `/v3/users/96cb72de833a4ef196a64f7b3ac157a4` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:07:44.980943 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] RBAC: Authorizing `identity:delete_user(user_id=96cb72de833a4ef196a64f7b3ac157a4)` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:07:44.982695 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:07:45.163573 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96cb72de833a4ef196a64f7b3ac157a4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:45.176546 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96cb72de833a4ef196a64f7b3ac157a4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:45.183605 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96cb72de833a4ef196a64f7b3ac157a4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:45.187232 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:07:45.250710 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96cb72de833a4ef196a64f7b3ac157a4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:45.251376 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.notifications [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96cb72de833a4ef196a64f7b3ac157a4'} {{(pid=81853) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:45.257712 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1470/2943] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 30 00:07:44 2022] DELETE /identity/v3/users/96cb72de833a4ef196a64f7b3ac157a4 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:07:45.487834 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:07:45.488886 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:07:45.505706 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60cc43-2955-48b9-b3de-4a10eaf80b26 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:07:45.529386 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:07:45.567620 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557208629c7f4826b594ef28cabef8ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291cfd89a4d34a9da9d99e5ca0c9d4dd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:07:45.568239 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:07:45.568513 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:07:45.568781 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] PATH_INFO: `/v3/projects/b9da58c01b6a48649611ebb67da31425` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:07:45.569978 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9da58c01b6a48649611ebb67da31425)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:07:45.571993 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:07:45.680093 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9da58c01b6a48649611ebb67da31425', 'request_id': 'req-5e43790a-aeb4-474c-91ea-13ea55b73c94'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:45.685632 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9da58c01b6a48649611ebb67da31425', 'request_id': 'req-5e43790a-aeb4-474c-91ea-13ea55b73c94'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:45.753766 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9da58c01b6a48649611ebb67da31425', 'request_id': 'req-5e43790a-aeb4-474c-91ea-13ea55b73c94'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:45.765189 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] The token cache is being invalidate because project b9da58c01b6a48649611ebb67da31425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:07:45.765480 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:45.766118 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5e43790a-aeb4-474c-91ea-13ea55b73c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:07:45.767624 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1474/2944] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 30 00:07:45 2022] DELETE /identity/v3/projects/b9da58c01b6a48649611ebb67da31425 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:07:55.303287 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:07:55.762081 dv-cinder-854272-4-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:10:19.719728 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] Authenticating user token {{(pid=81853) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:10:19.721125 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:10:19.864589 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550cd78-232d-4855-8853-9690ebc8669d admin admin] Validating token access rules against request {{(pid=81853) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:10:19.923193 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1dcf17-9777-4a6f-aecf-ae8aa9b4b855 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb770d4379e74bf79ad4280329c771c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eca691bde82145308c973e19aa9d1cbb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81853) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:10:19.923797 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1dcf17-9777-4a6f-aecf-ae8aa9b4b855 service placement] REQUEST_METHOD: `GET` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:10:19.924073 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1dcf17-9777-4a6f-aecf-ae8aa9b4b855 service placement] SCRIPT_NAME: `/identity` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:10:19.924848 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1dcf17-9777-4a6f-aecf-ae8aa9b4b855 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81853) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:10:19.926653 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.fernet_utils [None req-7c1dcf17-9777-4a6f-aecf-ae8aa9b4b855 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81853) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:10:19.937878 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1dcf17-9777-4a6f-aecf-ae8aa9b4b855 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:10:19.939557 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1dcf17-9777-4a6f-aecf-ae8aa9b4b855 service placement] RBAC: Authorization granted {{(pid=81853) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:10:19.950678 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: [pid: 81853|app: 0|req: 1471/2945] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 30 00:10:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:10:29.946044 dv-cinder-854272-4-iscsi devstack@keystone.service[81853]: DEBUG dbcounter [-] [81853] Writing DB stats keystone:SELECT=89 {{(pid=81853) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}